Each change in an order book is one of 5 event kinds with probabilities 1/2, 1/4, 1/8, 1/16 and 1/16. What is the Shannon entropy of the event kind in bits? Give a fraction.

Learn / Library / Probability

Entropy of book events

ProbabilityEasy

Each change in an order book is one of $5$ event kinds with probabilities $\frac{1}{2}$ , $\frac{1}{4}$ , $\frac{1}{8}$ , $\frac{1}{16}$ and $\frac{1}{16}$ . What is the Shannon entropy of the event kind in bits? Give a fraction.

Checking an answer needs JavaScript. The hint and the solution open without it.

Hint

Each probability is a power of $\frac{1}{2}$ , so every logarithm is a whole number.

Solution

The surprises $- log_{2} p$ are $1$ , $2$ , $3$ , $4$ and $4$ bits.
Weighting them gives $\frac{1}{2} \times 1 + \frac{1}{4} \times 2 + \frac{1}{8} \times 3 + \frac{1}{16} \times 4 + \frac{1}{16} \times 4 = \frac{15}{8}$ .

Answer: $\frac{15}{8}$

Next question Back to the library