A Markov chain is a probabilistic model where a system moves between a finite number of states at each step. The key property: the probability of moving to the next state depends only on the current state, not on history.
The transition matrix \(T\) encodes the probabilities of moving between states. For \(k\) states, \(T\) is a \(k \times k\) matrix where:
Each column of a transition matrix sums to 1 (column-stochastic).
Customers switch between Brand A and Brand B:
- If in A: 80% stay, 20% switch to B
- If in B: 30% switch to A, 70% stay
Check: column 1 sums to \(0.8 + 0.2 = 1\) ✓; column 2 sums to \(0.3 + 0.7 = 1\) ✓.
The state vector \(S_n\) gives the distribution of the system across states at step \(n\) (as proportions or counts). Initially \(S_0\) is given.
Initial market share: Brand A = 600 customers, Brand B = 400 customers.
After 1 period:
Interesting: market shares are unchanged! This is the steady-state (equilibrium) distribution.
As \(n \to \infty\), \(T^n S_0\) converges to a fixed vector \(S^*\) called the steady-state (or equilibrium) distribution, satisfying:
To find \(S^*\): solve the system \(TS^* = S^*\) (i.e., \((T - I)S^* = \mathbf{0}\)) together with the constraint that entries sum to 1. Or raise \(T\) to a large power on CAS.
KEY TAKEAWAY: Transition matrices model how populations, customers, or systems evolve over time. The steady state represents the long-run equilibrium regardless of the starting distribution.
VCAA FOCUS: Common question types: (1) calculate \(S_1\) or \(S_2\) by hand; (2) use CAS to find \(S_n\) for large \(n\); (3) identify or verify the steady-state vector; (4) interpret results in context.
