Recipe

Solve for a Markov chain's stationary distribution

Use when: you have a Markov chain's transition matrix P and need the distribution \boldsymbol\pi satisfying \boldsymbol\pi P=\boldsymbol\pi

Goal: the exact stationary distribution \boldsymbol\pi, with entries nonnegative and summing to 1

Taught in Stationary distributions

1. 1

   For each state, set the incoming-probability sum equal to that state's own \pi value.

   Example

   For P=\begin{pmatrix}0.8&0.2\\0.5&0.5\end{pmatrix}: the first coordinate gives 0.8\pi_1+0.5\pi_2=\pi_1.

   Why: this converts the single matrix equation \boldsymbol\pi P=\boldsymbol\pi into an ordinary system of linear equations in the unknowns \pi_i.

2. 2

   Because every row of P sums to 1, the equations from Step 1 are always dependent; simplify each one, then set aside one of them in favor of the normalization equation.

   Example

   0.8\pi_1+0.5\pi_2=\pi_1 simplifies to 0.5\pi_2=0.2\pi_1, so \pi_2=0.4\pi_1.

   Why: without dropping the redundant equation, the system would be one equation short of pinning down every unknown; the normalization equation supplies the missing one.

3. 3

   Solve the simplified relationship from Step 2 together with \pi_1+\pi_2+\cdots=1.

   Example

   \pi_1+0.4\pi_1=1, so \pi_1=\dfrac{5}{7} and \pi_2=\dfrac{2}{7}.

   Why: the eigenvector equation alone only fixes the ratio between the \pi_i's; the normalization equation is what fixes their actual scale.

4. 4

   Confirm no \pi_i came out negative.

   Example

   \dfrac{5}{7}\approx0.714 and \dfrac{2}{7}\approx0.286, both positive.

   Why: the eigenvector equation by itself can have solutions with negative entries, but only a nonnegative, normalized solution is an actual probability distribution.