Stick-breaking process
The stick-breaking process is a constructive representation of the Dirichlet process and related stochastic processes, in which a unit-length "stick" is recursively broken into fragments that serve as mixture weights. Each break point is drawn from a Beta distribution, and the remaining stick is broken again, producing an infinite sequence of weights that sum to one with probability one. This representation, introduced by Sethuraman in 1994, transforms an abstract measure-theoretic object into a sequential sampling procedure that can be implemented, simulated, and analyzed.
The Construction
Formally, let ν be a random measure drawn from a Dirichlet process with concentration parameter α and base distribution H. The stick-breaking construction represents ν as:
ν = Σ_{k=1}^∞ π_k δ_{θ_k}
where the weights π_k are generated by:
- Draw v_k ~ Beta(1, α) for k = 1, 2, ...
- Set π_1 = v_1
- Set π_k = v_k Π_{j=1}^{k-1} (1 - v_j) for k > 1
The atoms θ_k are drawn independently from the base distribution H. The construction guarantees that Σ π_k = 1 almost surely, and the resulting distribution is discrete with probability one — a property shared by all Dirichlet process draws.
The stick-breaking representation reveals why the Dirichlet process produces power-law behavior: the expected size of the k-th largest weight decays as k^{-1/α}, meaning that a small number of clusters captures most of the probability mass regardless of how many clusters exist in principle.
Generalizations Beyond the Dirichlet Process
The stick-breaking construction generalizes beyond the Dirichlet process. The Pitman-Yor process replaces the Beta(1, α) distribution with Beta(1 - d, α + kd), where d ∈ [0, 1) is a discount parameter. This modification produces a heavier tail in the weight distribution — the expected size of the k-th largest weight decays as k^{-1/d}, creating more small clusters and a richer tail structure than the Dirichlet process.
The stick-breaking representation has been extended to hierarchical Dirichlet processes, where multiple Dirichlet processes share a common base distribution, and to dependent Dirichlet processes, where the base distribution varies across covariates. In each case, the stick-breaking construction provides the computational backbone that makes inference tractable.
Why the Representation Matters
The stick-breaking process is not merely a mathematical trick. It is the bridge between the abstract world of measure theory and the concrete world of computation. Before Sethuraman's construction, the Dirichlet process was primarily a theoretical object — useful for proving theorems but difficult to simulate. The stick-breaking representation made it possible to write down explicit sampling algorithms, to implement Gibbs samplers, and to scale Bayesian nonparametrics to datasets with millions of observations.
The representation also clarifies the relationship between the concentration parameter α and the expected number of clusters. In the stick-breaking view, α controls the expected length of each break: higher α means smaller breaks, which means more fragments, which means more clusters. This intuition is invisible in the Chinese restaurant process or the Pólya urn representation, where the same parameter appears as a scaling factor in a seating probability.
The stick-breaking process reveals that the Dirichlet process is, at its core, a sequential decision procedure: at each step, decide how much of the remaining probability mass to allocate to the next cluster, then move on. This sequentiality is not an artifact of the representation. It is a property of the process itself, and it is why Bayesian nonparametrics scales gracefully with data: the model does not recompute all cluster weights when a new observation arrives; it simply considers whether the next observation warrants a new break.
The stick-breaking representation exposes a pattern that recurs across systems: the apparent complexity of an infinite object collapses into a simple iterative rule when viewed from the right angle. The Dirichlet process is not a distribution over distributions. It is a loop. The stick-breaking construction is the source code.