Likelihood ratio tests

Idea: Weigh 2 stories by their likelihoods

Divide the chance of the data under one story by the chance under the other.

• A signal hitting $\frac{3}{4}$ against a coin at $\frac{1}{2}$
• Each hit multiplies by $\frac{\frac{3}{4}}{\frac{1}{2}}=\frac{3}{2}$
• Each miss by $\frac{\frac{1}{4}}{\frac{1}{2}}=\frac{1}{2}$; arrangement counts cancel

Example: A signal hits 6 of 8

The signal hits $6$ of $8$ trades. What is the likelihood ratio of a $\frac{3}{4}$ hit rate against a coin flip?

1. Hit factor $\frac{3}{2}$, miss factor $\frac{1}{2}$
2. ${\left(\frac{3}{2}\right)}^6\times {\left(\frac{1}{2}\right)}^2=\frac{729}{256}$

Formula: The Neyman-Pearson lemma

$\text{reject}\text{when}LR>c$

For one null against one alternative, rejecting on a large ratio gives the most power at a fixed size.

Idea: Size and power

Size is the false alarm chance under the null. Power is the chance of catching a true alternative.

• Rule: reject at $7$ or more hits of $8$
• Size $\frac{9}{256}\approx 0.035$, power $\frac{24057}{65536}\approx 0.37$
• Small trials miss real signals

Quick check: The size of the test

Under a fair coin, what is the chance of $7$ or more hits in $8$ trades?

Formula: Wilks’ theorem

$-2\log \Lambda \approx {\chi }^2,df=\text{number}of\text{restrictions}$

Λ compares the best restricted fit with the best free fit. Under the null with a large sample it is close to χ².

Example: Testing a normal mean

$16$ daily returns average $\bar{x}=\frac{3}{2}$, with $\sigma =4$ known. The null is ${\mu }_0=0$. Find $-2\log \Lambda$.

1. It equals $\frac{n(\bar{x}-{\mu }_0{)}^2}{{\sigma }^2}$
2. $16\times \frac{\frac{9}{4}}{16}=\frac{9}{4}$

Idea: When the yardstick fails

The ${\chi }^2$ yardstick needs nested models, a large sample and null values off the edge.

• $2.25<3.84$: the example above does not reject
• A variance tested at zero sits on the edge
• Non-nested models get no yardstick at all

Common mistake: Reading the ratio as odds of truth

$LR\approx 2.85$ is not $2.85$ to $1$ odds that the vendor is honest.

Instead It multiplies your prior odds: from $\frac{1}{4}$ you reach about $0.71$, still against.

Idea: Say it in the room

State both hypotheses, fit both, and count restrictions for the degrees of freedom.

• Quote size and power together
• Short of the cut-off: not rejected, never proven true