The previous section's widget settled into a fixed split between Sunny and Rainy no matter where it started. That settling point has a name and an exact definition, and it's what lets a Markov source's long-run entropy rate (Chapter 8) even be a well-defined number.
Once a chain has been running long enough, the fraction of time it spends in each state stops changing step to step, even though the chain itself keeps moving. That fixed long-run split is the stationary distribution. It's not that the chain freezes — the weather still changes day to day — it's that if you tallied up a million days, the fraction that turned out sunny would settle down to one specific number, no matter which single day you happened to start counting from.
The eigenvector fact is the same idea, translated into linear algebra: \boldsymbol\pi (the Greek letter pi, in bold because it's a whole list of numbers — one probability per state, not a single number) is just that stationary distribution written as a vector. \boldsymbol\pi P = \boldsymbol\pi says "one more step doesn't move this particular distribution at all" — it's a fixed point of the whole process, the one input the transition matrix maps straight back to itself.
A probability distribution \boldsymbol\pi = (\pi_1, \pi_2, \ldots, \pi_r) over a Markov chain's states is a stationary distribution if applying one step of the chain leaves it unchanged: \boldsymbol\pi P = \boldsymbol\pi. Equivalently, \boldsymbol\pi is a left eigenvector of P for eigenvalue 1, normalized so its entries are nonnegative and sum to 1. For a chain where every state can reach every other state (an irreducible chain) and the chain isn't trapped in a strict cycle (aperiodic), this stationary distribution exists, is unique, and the chain's distribution converges to it from any starting point as the number of steps grows — which is exactly the settling-down the widget in the previous section showed happening.
Recipe Solve for a Markov chain's stationary distribution Use when: you have a Markov chain's transition matrix P and need the distribution \boldsymbol\pi satisfying \boldsymbol\pi P=\boldsymbol\pi Goal: the exact stationary distribution \boldsymbol\pi, with entries nonnegative and summing to 1
Quick recipe
-
1
For each state, set the incoming-probability sum equal to that state's own \pi value.
Example
For P=\begin{pmatrix}0.8&0.2\\0.5&0.5\end{pmatrix}: the first coordinate gives 0.8\pi_1+0.5\pi_2=\pi_1.
Why: this converts the single matrix equation \boldsymbol\pi P=\boldsymbol\pi into an ordinary system of linear equations in the unknowns \pi_i.
-
2
Because every row of P sums to 1, the equations from Step 1 are always dependent; simplify each one, then set aside one of them in favor of the normalization equation.
Example
0.8\pi_1+0.5\pi_2=\pi_1 simplifies to 0.5\pi_2=0.2\pi_1, so \pi_2=0.4\pi_1.
Why: without dropping the redundant equation, the system would be one equation short of pinning down every unknown; the normalization equation supplies the missing one.
-
3
Solve the simplified relationship from Step 2 together with \pi_1+\pi_2+\cdots=1.
Example
\pi_1+0.4\pi_1=1, so \pi_1=\dfrac{5}{7} and \pi_2=\dfrac{2}{7}.
Why: the eigenvector equation alone only fixes the ratio between the \pi_i's; the normalization equation is what fixes their actual scale.
-
4
Confirm no \pi_i came out negative.
Example
\dfrac{5}{7}\approx0.714 and \dfrac{2}{7}\approx0.286, both positive.
Why: the eigenvector equation by itself can have solutions with negative entries, but only a nonnegative, normalized solution is an actual probability distribution.
Common mistakes
- Trying to solve all r equations from Step 1 as if they were independent; one is always redundant given the others plus normalization.
- Forgetting the normalization equation entirely and reporting only a ratio between the entries, not actual probabilities.
- Accepting a solution with a negative entry as the stationary distribution instead of recognizing it as a sign of an arithmetic slip.
- Assuming a stationary distribution exists and is unique without the chain actually being irreducible and aperiodic.
Worked example
Solving for the stationary distribution directly
-
1
All four steps of the recipe above: find the stationary distribution of the two-state chain with transition matrix P = \begin{pmatrix}0.7 & 0.3\\0.3 & 0.7\end{pmatrix} (Sunny, Rainy) from the previous section.
-
2
Set up \boldsymbol\pi P = \boldsymbol\pi. Writing \pi_1 for Sunny and \pi_2 for Rainy, the first coordinate gives 0.7\pi_1 + 0.3\pi_2 = \pi_1, which simplifies to 0.3\pi_2 = 0.3\pi_1, so \pi_1=\pi_2. Combined with \pi_1+\pi_2=1,
-
3\boxed{\boldsymbol\pi = (0.5,\ 0.5)}
The symmetry of P (equal probability of switching either direction) makes this answer unsurprising — a genuinely asymmetric chain needs the full linear system solved, not a symmetry shortcut.
An asymmetric chain — no symmetry shortcut this time
Press Play and watch where this one settles. It won't be 50/50 — predict which state it favors before checking the readout.
Loading interactive…
Worked example
Two different starting points, same destination
-
1
For the chain above (working machine, 90% stays working, 10% breaks; broken machine, 40% gets fixed, 60% stays broken), does the starting distribution affect where the chain settles?
-
2
Run the chain from "always starts working" and from "always starts broken" — both converge to the same stationary distribution as the number of steps grows, exactly the uniqueness-and-convergence guarantee the technical box states for an irreducible, aperiodic chain. This particular chain is irreducible (each state can reach the other) and aperiodic (no forced strict cycle), so the guarantee applies.
-
3\boxed{\text{No — the long-run split is the same regardless of where the chain started.}}
Confirm pi P = pi numerically
Step through P\pi = (0.5, 0.5) for the Sunny/Rainy chain — confirm the result is exactly (0.5, 0.5) again, unchanged.
Loading interactive…
Does a stationary distribution exist and is it unique?
Sort each chain description by whether it has a unique stationary distribution.
Loading interactive…
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
What this task checks
Structure checked automatically. Whether your ideas are convincing isn't graded, so reread with the checklist.
The compiler says
A model answer
Correct answer
Also accepted:
Why
Why
Where this goes next
Chapter 8's entropy rate for a Markov source, H_\infty(X) = -\sum_j \pi_j \sum_i P_{ij}\log P_{ij}, is a weighted average of the uncertainty at each state — weighted by exactly this stationary distribution. Without \boldsymbol\pi pinned down first, that formula has no fixed set of weights to average over.
Check your understanding (counts toward your progress)
A two-state chain has P=\begin{pmatrix}0.6&0.4\\0.4&0.6\end{pmatrix}. By symmetry, find \pi_1 (the first coordinate of the stationary distribution).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
Since P treats both states symmetrically (equal probability of switching either direction), the stationary distribution is uniform: \pi_1=\pi_2=0.5.
Correct answer
Also accepted:
Why
Compare the two rows of P — if swapping the labels of the two states leaves the matrix looking identical, the stationary distribution can't favor either state over the other.
A chain has two separate groups of states that never transition into each other. What happens to its stationary distribution?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Uniqueness specifically requires irreducibility (every state reachable from every other) — this chain violates that.
A stationary distribution still exists on each separate piece — it's the UNIQUENESS that fails, not existence.
Reducibility has nothing to do with the Markov (unit-memory) property — that's a separate structural question entirely.
Why
Without irreducibility, each disconnected group has its own stationary distribution, and any weighted combination of them is also stationary for the whole chain — uniqueness specifically requires irreducibility.
Recall which structural property the uniqueness guarantee for a stationary distribution actually depends on, and check whether two disconnected groups of states have it.
Each disconnected group can support its OWN stationary distribution on its own states — think about what happens when you combine two separately-valid stationary distributions with different weights.
A stationary distribution \boldsymbol\pi of a Markov chain with transition matrix P satisfies:
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
That would make \boldsymbol\pi an eigenvector for eigenvalue 0, not a fixed point of the chain.
A probability distribution's entries sum to 1, not 0.
Why
\boldsymbol\pi P=\boldsymbol\pi says one more step of the chain leaves the distribution unchanged — the definition of stationary.
Practice: Stationary distributions (5 problems · doesn’t affect your progress)
Which statement below is NOT an equivalent way of saying \boldsymbol\pi is a stationary distribution of P?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
This IS the defining equation of a stationary distribution.
This IS an equivalent restatement — a stationary distribution is exactly a left eigenvector for eigenvalue 1.
This also follows — if one step leaves \boldsymbol\pi unchanged, so does any number of further steps.
Why
Summing to 1 is required of EVERY probability distribution, stationary or not — it alone says nothing about whether a step of the chain leaves it fixed.
\boldsymbol\pi is any probability distribution summing to 1
For P=\begin{pmatrix}0.6&0.5\\0.4&0.5\end{pmatrix} (columns = current state, rows = next state), compute the first coordinate of \boldsymbol\pi P for \boldsymbol\pi=(0.5,0.5).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
(\boldsymbol\pi P)_1 = \pi_1 P_{11}+\pi_2 P_{12} = 0.5(0.6)+0.5(0.5)=0.55 — since this doesn't equal \pi_1=0.5, (0.5,0.5) is not the stationary distribution for this (non-symmetric) matrix.
Correct answer
Also accepted:
Why
Find the actual stationary distribution's first coordinate, \pi_1, for P=\begin{pmatrix}0.6&0.5\\0.4&0.5\end{pmatrix}.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
\pi_1 = 0.6\pi_1+0.5\pi_2 \Rightarrow 0.4\pi_1=0.5\pi_2 \Rightarrow \pi_1=1.25\pi_2. With \pi_1+\pi_2=1: \pi_2=4/9, \pi_1=5/9\approx 0.5556.
Correct answer
Also accepted:
Why
Set up \pi_1 = 0.6\pi_1+0.5\pi_2 from the first coordinate, simplify, then bring in \pi_1+\pi_2=1 to solve for both unknowns.
A two-state chain alternates deterministically: from A it always goes to B next, and from B it always goes to A next (P=\begin{pmatrix}0&1\\1&0\end{pmatrix}). This chain is irreducible (each state reaches the other). Starting from 'always A', does the chain's distribution converge to a fixed stationary distribution as steps increase?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Irreducibility alone isn't the full guarantee — aperiodicity is the other required condition, and this chain is strictly periodic (period 2).
A stationary distribution does exist here — (0.5,0.5) solves \boldsymbol\pi P=\boldsymbol\pi — it's convergence FROM an arbitrary start that fails.
It never settles at all, let alone after one step — it keeps flipping between the two extremes forever.
Why
This chain is irreducible but periodic (period 2), violating the aperiodicity half of the existence-uniqueness-convergence guarantee — a stationary distribution exists on paper, but starting from all-mass-on-A, the distribution never settles; it just keeps flipping between (1,0) and (0,1).
No — it keeps alternating between all-A and all-B forever, never settling, even though a stationary distribution (0.5,0.5) exists on paper
A chain has an absorbing state (once entered, never left) plus other transient states leading into it. Is \boldsymbol\pi with all its probability mass on that one absorbing state (and 0 everywhere else) a valid stationary distribution?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Nothing in the definition \boldsymbol\pi P=\boldsymbol\pi requires every state to get positive probability.
Symmetry has nothing to do with it — the absorbing structure alone is what makes this work.
The argument holds for any number of transient states feeding into the one absorbing state, not just two total.
Why
Putting all probability on the absorbing state means every subsequent step keeps all the mass right there too — \boldsymbol\pi P=\boldsymbol\pi holds exactly, though because this chain (with transient states feeding in) is reducible, this isn't the unique stationary distribution in general, just one valid solution.
Yes — one step from the absorbing state stays in the absorbing state, so \boldsymbol\pi P=\boldsymbol\pi holds trivially