Conjugate priors

Idea: A pair of numbers instead of a curve

With a conjugate prior the posterior stays in the prior’s family, so an update changes a couple of numbers.

• No curve to store or normalise
• Order does not matter: just keep adding

Formula: Beta and binomial

$\mathrm{Beta}(a,b)\to \mathrm{Beta}(a+x,b+n-x)$

After x fills in n orders. Read a and b as fills and misses you have already seen.

Example: Updating a fill rate

Prior $\mathrm{Beta}(4,6)$, worth $10$ orders. Then $12$ of $20$ orders fill. What is the posterior mean?

1. Posterior $\mathrm{Beta}(16,14)$
2. Mean $\frac{16}{30}=\frac{8}{15}$

Idea: A weighted average

The posterior mean averages the raw rate and the prior mean; the data weigh $\frac{n}{n+a+b}$.

• Here $\frac{20}{30}=\frac{2}{3}$
• Check: $\frac{2}{3}\times \frac{3}{5}+\frac{1}{3}\times \frac{2}{5}=\frac{8}{15}$

Common mistake: Same mean, same prior

$\mathrm{Beta}(3,7)$ and $\mathrm{Beta}(30,70)$ both have mean $\frac{3}{10}$, yet data move them very differently.

Instead After $5$ fills in $10$: $\frac{3+5}{10+10}=\frac{2}{5}$ but $\frac{30+5}{100+10}=\frac{7}{22}$. State $a+b$.

Formula: Gamma and Poisson

$\mathrm{Gamma}(a,b)\to \mathrm{Gamma}(a+s,b+t)$

After s events in t days: a pseudo-events over b pseudo-days, mean (a + s)/(b + t).

Quick check: A disconnect rate

Prior $\mathrm{Gamma}(3,2)$ for daily disconnects. Over $5$ days you count $11$. What is the posterior mean rate?

Formula: Normal and normal: precisions add

$\frac{1}{V}=\frac{1}{v}+\frac{n}{{\sigma }^2}$

Precision is 1 over a variance. The posterior mean averages m and x̄ weighted by their precisions.

Example: A normal update

Prior $N(1,6)$ for the mean, ${\sigma }^2=12$, $n=4$, $\bar{x}=7$. Find the posterior mean.

1. Precision $\frac{1}{6}+\frac{4}{12}=\frac{1}{2}$, so $V=2$
2. Mean $2\times \left(\frac{1}{6}\times 1+\frac{4}{12}\times 7\right)=5$

Idea: When to stop using them

An honest prior with $2$ humps, or fat tails, fits no neat family: use a grid or a sampler.

• A conjugate model still sanity-checks the heavy method