Hidden Markov Models
A hidden Markov model (HMM) is a statistical model in which a system is assumed to be a Markov process with unobservable (hidden) states that generate observable outcomes through a probabilistic emission function. The model is hidden because the state sequence is not directly visible; only the emissions are. HMMs are the canonical framework for modeling sequential data where the underlying generative structure must be inferred from surface observations.
The HMM is defined by five elements:
- N, the number of hidden states
- M, the number of distinct observable symbols
- A, the state transition matrix (N × N)
- B, the emission probability matrix (N × M)
- π, the initial state distribution
The three canonical problems — evaluation (what is the probability of an observed sequence?), decoding (what is the most likely hidden state sequence?), and learning (what are the parameters that best explain the data?) — are solved by the forward algorithm, the Viterbi algorithm, and the Baum-Welch algorithm respectively.
Applications Across Domains
HMMs have been deployed in virtually every field that deals with sequential inference. In speech recognition, hidden states represent phonemes and emissions represent acoustic features. In bioinformatics, hidden states represent functional categories (coding vs. non-coding) and emissions represent DNA bases. In natural language processing, hidden states represent syntactic categories and emissions represent words or morphemes. In finance, hidden states represent market regimes (bull, bear, volatile) and emissions represent price movements.
This cross-domain applicability is not accidental. The HMM captures a structural pattern that recurs whenever a system generates observable traces through an internal state machine that evolves with Markovian memorylessness. The pattern is so general that HMMs have been described as the "Fourier transform of sequential inference" — a universal decomposition tool for time-series data.
The HMM and Causality
HMMs occupy an interesting position in the philosophy of causation. The hidden states are not causes in the interventionist sense — they cannot be manipulated directly. They are latent variables, constructs that summarize the statistical regularities of the emission sequence. The Viterbi algorithm finds the most probable state sequence, but probability is not causality. An HMM that assigns high probability to a particular hidden state path does not claim that the hidden states caused the emissions; it claims that the emissions are best explained by that path.
This distinction matters. HMMs are sometimes criticized as "merely descriptive" or "unfalsifiable" because the hidden states are not independently observable. But this criticism misunderstands the model's purpose. HMMs are not theories of hidden causes; they are compression algorithms. They compress the statistical structure of observation sequences into a compact state-transition representation. The hidden states are bookkeeping devices, not ontological claims.
The hidden Markov model is the computational form of a very old epistemological move: when you cannot observe the thing that generates the pattern, infer the simplest machine that could have produced it. The HMM does not discover hidden reality. It constructs a compact fiction that predicts the observable world. That fiction may or may not correspond to anything real — but if it predicts well, it is useful, and usefulness is the only criterion that matters for a model of this kind.