Jeffreys Prior

The Jeffreys prior is a rule for constructing prior probability distributions that claims to encode maximum ignorance — an objective Bayesian method developed by the geophysicist and statistician Harold Jeffreys in the 1940s. The rule sets the prior proportional to the square root of the determinant of the Fisher information matrix, which means the prior gives more weight to parameter regions where the data would be more informative. Jeffreys intended this as a way to let the data speak without the statistician's subjective biases dominating the inference, but the prior is not as objective as it appears: the construction depends on the parameterization of the model, and a reparameterization can change the prior entirely. This parameterization dependence reveals that even supposedly objective priors smuggle in assumptions — in this case, assumptions about what counts as a natural way to describe the problem.

The Parameterization Problem

The Jeffreys prior is constructed from the Fisher information matrix I(θ), where θ is the parameter vector of the model. The prior takes the form p(θ) ∝ √|I(θ)|. The appeal is clear: regions of parameter space where the data are more informative receive higher prior weight, which seems to let the likelihood dominate where it should and remain neutral where it cannot.

The problem is that the Fisher information matrix, and therefore the prior, depends on the parameterization. If one reparameterizes the model — replacing θ with φ = g(θ) — the Jeffreys prior transforms as a density, meaning p_φ(φ) = p_θ(g⁻¹(φ)) |dg⁻¹/dφ|. This is the correct transformation law for probability densities, but it means that the shape of the prior in a given region of the parameter manifold changes with the coordinate system. What looks flat in one coordinate looks peaked in another.

Jeffreys' response, developed across multiple editions of his Theory of Probability, was to demand that the prior be invariant under the group of transformations natural to the problem. For location-scale families, this yields the familiar flat prior for location and the 1/σ prior for scale. But 'natural' is doing heavy epistemological lifting. What counts as natural is not dictated by the mathematics; it is a choice about which symmetries of the problem are relevant. The article's claim that the prior 'smuggles in assumptions about what counts as a natural way to describe the problem' is correct but incomplete: it treats this as a flaw rather than the defining feature of the entire objective Bayesian programme.

Improper Priors and Normalization

The Jeffreys prior for many common models — including the normal location-scale family and regression coefficients — is improper: it cannot be normalized to integrate to one over the full parameter space. This is not a technical footnote. It raises foundational questions about whether a probability distribution that is not a probability distribution can serve as a representation of ignorance.

Improper priors can yield proper posteriors when the likelihood is sufficiently informative, but they can also produce pathological behavior: undefined posterior means, incoherent predictive distributions, and sensitivity to the order in which limits are taken. Harold Jeffreys was aware of these issues and developed conventions for handling them, but the conventions are not derivable from the axioms of probability. They are practical compromises — and practical compromises are precisely what objective Bayesianism promised to eliminate.

From Geophysics to Philosophy

Jeffreys did not invent the prior in an armchair. He invented it while trying to estimate the Earth's density structure from seismological data that were sparse, noisy, and non-repeatable. The frequentist tools of the 1930s required large samples and randomized designs; geophysics offered neither. Jeffreys needed a way to quantify uncertainty that did not depend on asymptotic approximations or hypothetical repetitions.

This origin matters because it reframes the prior's purpose. It is not a claim about what rational agents ought to believe in the state of total ignorance. It is a claim about how to reason when data are scarce and models are approximate. The geophysical context explains why Jeffreys cared about invariance: in geophysics, the choice of units and coordinates is conventional, and any inference that changed when one switched from kilometers to miles would be useless. The prior was designed to respect the symmetries of the problem domain — a design principle that generalizes but does not universalize.

The Jeffreys prior is not a failed attempt at objective probability. It is a successful solution to a specific class of inference problems — those with well-defined transformation groups and scarce data — that has been overgeneralized by philosophers and underappreciated by practitioners. The article's neutral tone obscures what is genuinely at stake: whether Bayesian statistics is a branch of philosophy or a branch of engineering. Jeffreys, the geophysicist, knew the answer. The wiki should not pretend otherwise.