Shrinkage and empirical Bayes

Idea: Shrink toward what you know

A strategy wins $9$ of $10$ trades; similar ones win $\frac{3}{5}$. Do you size it as a $90\%$ winner?

• $10$ trades say little
• What you know about similar strategies is evidence too
• Shrinkage turns that instinct into arithmetic

Formula: The shrinkage form

$T=(1-\alpha )\times \hat{\theta }+\alpha \times {\mu }_0$

Every shrinkage estimator is a weighted average of the raw estimate and a target μ₀.

Example: A prior worth 20 trades

The book’s prior is $\mathrm{Beta}(12,8)$. The new strategy wins $9$ of $10$ trades. What is the posterior mean?

1. Posterior $\mathrm{Beta}(21,9)$
2. Mean $\frac{21}{30}=\frac{7}{10}$

Idea: Evidence earns its way past the prior

The prior’s weight is its size over the total: $\frac{20}{20+10}=\frac{2}{3}$.

• Check: $\frac{2}{3}\times \frac{3}{5}+\frac{1}{3}\times \frac{9}{10}=\frac{7}{10}$
• With $100$ trades: $\frac{20}{120}=\frac{1}{6}$

Formula: Why bias can help

$\mathrm{MSE}(\alpha )=(1-\alpha {)}^2{s}^2+{\alpha }^2{\delta }^2$

Shrinking cuts variance and adds bias αδ. The best weight is α = s²/(s² + δ²).

Quick check: The error after shrinking

A raw estimate has standard error $s=2$; the truth sits $\delta =1$ from the target. What is the mean squared error at the best $\alpha$?

Example: The group sets the prior

Desk scores $1$, $3$, $7$, $9$ with known noise $v=\frac{10}{3}$. Where does the top desk move?

1. Centre $5$, $S^2=\frac{16+4+4+16}{3}=\frac{40}{3}$
2. ${\hat{\tau }}^2=\frac{40}{3}-\frac{10}{3}=10$, keep $\frac{10}{10+\frac{10}{3}}=\frac{3}{4}$
3. Top desk: $5+\frac{3}{4}\times 4=8$

Idea: Empirical Bayes

With many similar units, the spread between them sets the prior: no opinion needed.

• A negative ${\hat{\tau }}^2$? Set it to zero and shrink fully
• With $3$ or more means, shrinking together lowers total error

Common mistake: Expecting every estimate to improve

Shrinkage lowers total error across the group; a truly exceptional unit is pulled in too.

Instead Shrink toward the right population, and treat an empirical prior as an estimate: widen intervals.

Idea: Say it in the room

Name the raw estimate, the target, and where the weight comes from.

• Weight: noise over noise plus signal
• Biggest gains on small samples
• Raw league tables flatter the top names