A Markov chain is a probabilistic model where a system moves between a finite number of states at each step. The key property: the probability of moving to the next state depends only on the current state, not on history.
The transition matrix $T$ encodes the probabilities of moving between states. For $k$ states, $T$ is a $k \times k$ matrix where:
$$t_{ij} = P(\text{move to state } i \mid \text{currently in state } j)$$
Each column of a transition matrix sums to 1 (column-stochastic).
Customers switch between Brand A and Brand B:
- If in A: 80% stay, 20% switch to B
- If in B: 30% switch to A, 70% stay
$$T = \begin{pmatrix} 0.8 & 0.3 \ 0.2 & 0.7 \end{pmatrix}$$
Check: column 1 sums to \$0.8 + 0.2 = 1$ ✓; column 2 sums to \$0.3 + 0.7 = 1$ ✓.
The state vector $S_n$ gives the distribution of the system across states at step $n$ (as proportions or counts). Initially $S_0$ is given.
$$S_n = T \cdot S_{n-1} = T^n \cdot S_0$$
Initial market share: Brand A = 600 customers, Brand B = 400 customers.
$$S_0 = \begin{pmatrix} 600 \ 400 \end{pmatrix}$$
After 1 period:
$$S_1 = TS_0 = \begin{pmatrix} 0.8 & 0.3 \ 0.2 & 0.7 \end{pmatrix}\begin{pmatrix} 600 \ 400 \end{pmatrix} = \begin{pmatrix} 0.8(600)+0.3(400) \ 0.2(600)+0.7(400) \end{pmatrix} = \begin{pmatrix} 600 \ 400 \end{pmatrix}$$
Interesting: market shares are unchanged! This is the steady-state (equilibrium) distribution.
As $n \to \infty$, $T^n S_0$ converges to a fixed vector $S^$ called the steady-state (or equilibrium*) distribution, satisfying:
$$T S^ = S^ \quad \text{and} \quad \text{entries of } S^* \text{ sum to } 1 \text{ (for proportions)}$$
To find $S^$: solve the system $TS^ = S^$ (i.e., $(T - I)S^ = \mathbf{0}$) together with the constraint that entries sum to 1. Or raise $T$ to a large power on CAS.
KEY TAKEAWAY: Transition matrices model how populations, customers, or systems evolve over time. The steady state represents the long-run equilibrium regardless of the starting distribution.
VCAA FOCUS: Common question types: (1) calculate $S_1$ or $S_2$ by hand; (2) use CAS to find $S_n$ for large $n$; (3) identify or verify the steady-state vector; (4) interpret results in context.
