Jeffreys prior

Jeffreys prior is a method for constructing objective prior distributions in Bayesian inference, proposed by the geophysicist and statistician Harold Jeffreys in 1946. It is designed to be minimally informative in a precisely defined sense: the prior is proportional to the square root of the determinant of the Fisher information matrix, which means it assigns more probability to parameter regions where the data would be more informative.

The rationale is elegant. A prior that is uniform in one parameterization is not uniform in another — the uniform prior for a variance is not uniform for a standard deviation. Jeffreys solved this by constructing a prior that is invariant under reparameterization: if you transform the parameter, the Jeffreys prior transforms in a way that preserves its information-theoretic character. This invariance makes it a genuine default prior rather than a convenient choice.

Jeffreys prior is not always proper — it may not integrate to one over the entire parameter space — and it can produce counterintuitive results in high dimensions, where it tends to concentrate probability in ways that favor simpler models. In Bayesian model selection, this property connects Jeffreys prior to automatic complexity penalization, blurring the boundary between prior choice and model comparison.

The deeper significance of Jeffreys prior is that it represents an attempt to extract a unique, data-driven prior from the likelihood itself, collapsing the distinction between what we believe before seeing data and what the data model tells us about where information will be found. Whether this collapse is a methodological convenience or a philosophical confusion remains debated.

Jeffreys prior is the formal expression of a seductive idea: that we can derive what we should believe from the structure of what we are trying to learn. The idea is seductive because it promises to eliminate subjectivity from Bayesian inference — to make the prior 'objective' by grounding it in the mathematics of the likelihood. But the promise is hollow. The Fisher information matrix depends on the model, and the model is chosen, not discovered. Jeffreys prior is not objective; it is objective relative to a model that is itself a contingent choice. It replaces the subjectivity of prior belief with the subjectivity of model specification — and the latter is often less visible and therefore more dangerous.