Riemannian manifold HMC

Riemannian manifold Hamiltonian Monte Carlo (RMHMC) is an extension of Hamiltonian Monte Carlo that replaces the Euclidean mass matrix with the Fisher information metric, making the Hamiltonian dynamics adaptive to the local curvature of the posterior distribution. Developed by Girolami and Calderhead in 2011, RMHMC recognizes that the parameter space of a statistical model is not a flat Euclidean space but a Riemannian manifold, and that proposals which respect this geometry will explore the distribution more efficiently than proposals that ignore it.

In standard HMC, the kinetic energy is defined with a fixed mass matrix that must be tuned or estimated from preliminary samples. In RMHMC, the mass matrix becomes the Fisher information metric G(θ), which varies with position and encodes the local geometry of the posterior. The Hamiltonian equations become geodesic equations on the manifold, and the leapfrog integrator must be replaced by a generalized leapfrog scheme that accounts for the metric's position dependence. This added complexity — each step requires computing the metric and its derivatives — is the cost of respecting the geometry.

RMHMC is most beneficial for distributions with strong parameter correlations or varying curvature, where standard HMC's fixed mass matrix is either too conservative in some directions or too reckless in others. In hierarchical models, mixture models, and neural network posteriors, the Fisher metric can reveal directions of high variance that are invisible to the Euclidean eye. The method has also been generalized to Lagrangian Monte Carlo and to stochastic gradient variants that approximate the metric from minibatches.

Riemannian manifold HMC is the natural next step in a sequence that began with random walks and progressed to Hamiltonian dynamics. But it has been curiously underadopted in practice, despite its theoretical advantages. The reason is not computational cost alone — it is that the field of Bayesian computation has not yet developed the geometric intuition required to diagnose when a posterior's curvature demands manifold-aware sampling. Most practitioners remain stuck on flat spaces, not because the manifold is inaccessible but because the map is unreadable. The failure of RMHMC to achieve mainstream adoption is less a failure of the algorithm than a failure of the field's education.