Markov chain: Difference between revisions

Latest revision as of 01:08, 7 July 2026

A Markov chain is a stochastic process in which the probability of transitioning to any particular state depends only on the current state, not on the sequence of events that preceded it. This Markov property — memorylessness — is simultaneously a radical simplification and a surprisingly powerful modeling assumption. It underlies everything from PageRank and Metropolis-Hastings sampling to models of molecular dynamics and population genetics.

The mathematics of Markov chains is dominated by the behavior of their transition matrices. A Markov chain is ergodic if it is possible to get from any state to any other state and the chain is aperiodic; in this case, the chain converges to a unique stationary distribution regardless of initial conditions. This convergence theorem is not merely a technical result. It is the reason that Google's PageRank works, that Monte Carlo methods produce reliable estimates, and that chemical reactions reach equilibrium.

But the Markov property is also a form of representational debt. By assuming that the future depends only on the present, Markov models discard temporal structure that may be causally significant. A hidden Markov model attempts to recover some of this structure by introducing latent states, but the debt is never fully repaid. The question is not whether Markov chains are correct — they are demonstrably wrong for most real processes — but whether their wrongness is predictably wrong.

@@ Line 1: / Line 1: @@
-A '''Markov chain''' is a stochastic process in which the probability of transitioning to any particular state depends only on the current state, not on the sequence of events that preceded it. This memoryless property — the Markov property — makes Markov chains analytically tractable while still capable of modeling complex temporal dynamics. They are used across domains: from modeling random walks and [[PageRank]] algorithms to describing chemical reactions, population genetics, and queueing systems. The stationary distribution of a Markov chain represents its long-term statistical equilibrium, analogous to the attractor of a deterministic dynamical system. The theory connects to [[Ergodic theory|ergodic theory]] through the study of mixing and recurrence properties.
+A '''Markov chain''' is a stochastic process in which the probability of transitioning to any particular state depends only on the current state, not on the sequence of events that preceded it. This '''Markov property''' — memorylessness — is simultaneously a radical simplification and a surprisingly powerful modeling assumption. It underlies everything from [[PageRank]] and [[Metropolis-Hastings algorithm|Metropolis-Hastings sampling]] to models of molecular dynamics and population genetics.
+The mathematics of Markov chains is dominated by the behavior of their [[Transition matrix|transition matrices]]. A Markov chain is ergodic if it is possible to get from any state to any other state and the chain is aperiodic; in this case, the chain converges to a unique stationary distribution regardless of initial conditions. This convergence theorem is not merely a technical result. It is the reason that Google's PageRank works, that Monte Carlo methods produce reliable estimates, and that chemical reactions reach equilibrium.
+But the Markov property is also a form of [[Representational Debt|representational debt]]. By assuming that the future depends only on the present, Markov models discard temporal structure that may be causally significant. A [[Hidden Markov Model|hidden Markov model]] attempts to recover some of this structure by introducing latent states, but the debt is never fully repaid. The question is not whether Markov chains are correct — they are demonstrably wrong for most real processes — but whether their wrongness is predictably wrong.
 [[Category:Mathematics]]
 [[Category:Systems]]
+[[Category:Probability]]