Entropy in inference

Idea: Entropy counts what you do not know

$8$ equally likely causes of a fill drop: ${\log }_{2}8=3$ bits of doubt.

• Tests are ranked by how much they remove
• Priors should add nothing you cannot justify
• A bonus pays a learner to keep some

Example: Which test runs first

$4$ causes sit in the engine, $2$ in the network, $2$ in the data. A test names the group. How many bits does it remove on average?

1. Left after: $2$ bits for the engine, $1$ otherwise
2. Average $\frac{1}{2}\times 2+\frac{1}{4}\times 1+\frac{1}{4}\times 1=\frac{3}{2}$
3. Gain $3-\frac{3}{2}=\frac{3}{2}$

Idea: Rank tests by bits per hour

A good test splits the probability evenly among its outcomes.

• A test that almost always says the same teaches little
• Expected gain is the mutual information of cause and result

Formula: Maximum entropy

$\hat{p}=\mathrm{argmax}H(p),\text{subject}to\text{the}\text{known}\text{facts}$

Of all laws that fit the facts, take the most uncertain: anything else assumes more.

Idea: What the facts give

The maximum entropy law depends on what you know.

• Only the outcomes: uniform
• Group totals: uniform inside each group
• A mean: $p(k)\propto r^k$, tilting steadily

Quick check: A maximum entropy fill count

A strategy gets $0$, $1$ or $2$ fills with mean $\frac{5}{13}$. Take the maximum entropy law. What is $P(0)$?

Formula: Paying a strategy to stay curious

$J(\pi )=E[r]+\beta H(\pi )$

Score a policy by its reward plus β times its entropy, so it keeps exploring.

Example: When spreading flow pays

Venues pay $6$, $4$, $2$ ticks. Policy A always takes the first; B routes $\frac{1}{2}$, $\frac{1}{4}$, $\frac{1}{4}$. At what $\beta$ do they tie?

1. B earns $\frac{1}{2}\times 6+\frac{1}{4}\times 4+\frac{1}{4}\times 2=\frac{9}{2}$
2. B’s entropy is $\frac{3}{2}$, A’s is $0$
3. Tie: $\beta =\frac{6-\frac{9}{2}}{\frac{3}{2}}=1$

Common mistake: Uncertainty is not a forecast

A maximum entropy law states what you do not know. With wrong constraints the answer is wrong.

Instead A curious policy is not a profitable one: the bonus buys data at a cost in reward. Lower $\beta$ as you learn.