Learn / Library / Linear algebra

Machine uptime

Linear algebraHard

A machine is checked every hour and is either up (state $1$ ) or down (state $2$ ). Its state follows a Markov chain with transition matrix $P = [\frac{3}{4} \frac{1}{2} \frac{1}{4} \frac{1}{2}]$ (rows separated by semicolons), where the entry in row $i$ , column $j$ is the probability of moving from state $i$ to state $j$ . The machine is up now. What is the probability that it is up at the check $3$ hours from now? Give a fraction in lowest terms.

Checking an answer needs JavaScript. The hint and the solution open without it.

Hint

Use the eigenvalues of $P$ : the answer is the long-run share plus a correction that shrinks geometrically.

Solution

The eigenvalues of $P$ are $1$ and $\frac{3}{4} + \frac{1}{2} - 1 = \frac{1}{4}$ , because their sum is the trace.
The stationary vector solves $π_{1} \times \frac{1}{4} = π_{2} \times \frac{1}{2}$ , so $π = (\frac{2}{3}, \frac{1}{3})$ .
So the top left entry of $P^{n}$ is $\frac{2}{3} + c \times (\frac{1}{4})^{n}$ for a fixed $c$ , and since it equals $1$ at $n = 0$ , we get $c = \frac{1}{3}$ .
For $n = 3$ this is $\frac{2}{3} + \frac{1}{3} \times \frac{1}{64} = \frac{129}{192} = \frac{43}{64}$ .

Answer: $\frac{43}{64}$

Next question Back to the library