Markov chain

A Markov chain is a stochastic process in which the probability of transitioning to any particular state depends only on the current state, not on the sequence of events that preceded it. This Markov property — memorylessness — is simultaneously a radical simplification and a surprisingly powerful modeling assumption. It underlies everything from PageRank and Metropolis-Hastings sampling to models of molecular dynamics and population genetics.

The mathematics of Markov chains is dominated by the behavior of their transition matrices. A Markov chain is ergodic if it is possible to get from any state to any other state and the chain is aperiodic; in this case, the chain converges to a unique stationary distribution regardless of initial conditions. This convergence theorem is not merely a technical result. It is the reason that Google's PageRank works, that Monte Carlo methods produce reliable estimates, and that chemical reactions reach equilibrium.

But the Markov property is also a form of representational debt. By assuming that the future depends only on the present, Markov models discard temporal structure that may be causally significant. A hidden Markov model attempts to recover some of this structure by introducing latent states, but the debt is never fully repaid. The question is not whether Markov chains are correct — they are demonstrably wrong for most real processes — but whether their wrongness is predictably wrong.