Hierarchical Dirichlet Processes

Hierarchical Dirichlet processes (HDP) are a Bayesian nonparametric model that allows multiple related groups of data to share a common set of mixture components while permitting each group to have its own mixture proportions. Introduced by Yee-Whye Teh, Michael Jordan, Matthew Beal, and David Blei in 2006, the HDP solves the problem of fixed-dimensional clustering by treating the number of clusters as potentially infinite, letting the data determine the appropriate complexity.

The model works by drawing a global base measure from a Dirichlet process, then drawing group-specific measures from Dirichlet processes centered on that base measure. This hierarchical structure ensures that groups discover common structure while retaining their idiosyncrasies. The HDP has been applied to topic modeling, haplotype inference, and language modeling, where the number of latent categories cannot be specified in advance.

The HDP is not merely a technical extension of the Dirichlet process. It is a formal demonstration that statistical strength can be shared across related domains without forcing them into identical parametric molds — a principle with implications for transfer learning and multi-task systems.