Each day has one of 3 outcomes, drawn independently. The true law P gives them probabilities 1/4, 1/4, 1/2, and a rival model Q gives 1/2, 1/4, 1/4. Each day you add log₂ (p(x)/q(x)) to a score, where x is the outcome of the day. What is the expected score after 10 days? Give a fraction.

Learn / Library / Statistics

Evidence over many days

StatisticsMedium

Each day has one of $3$ outcomes, drawn independently. The true law $P$ gives them probabilities $\frac{1}{4}$ , $\frac{1}{4}$ , $\frac{1}{2}$ , and a rival model $Q$ gives $\frac{1}{2}$ , $\frac{1}{4}$ , $\frac{1}{4}$ . Each day you add $log_{2} (\frac{p ( x )}{q ( x )})$ to a score, where $x$ is the outcome of the day. What is the expected score after $10$ days? Give a fraction.

Checking an answer needs JavaScript. The hint and the solution open without it.

Hint

The expected daily score is the KL divergence.

Solution

The daily scores are $- 1$ , $0$ and $1$ .
The expected daily score is $\frac{1}{4} \times (- 1) + \frac{1}{4} \times 0 + \frac{1}{2} \times 1 = \frac{1}{4}$ .
Expectations add over days, so the score is $10 \times \frac{1}{4} = \frac{5}{2}$ .

Answer: $\frac{5}{2}$

Next question Back to the library