Jeffreys prior: Difference between revisions

Latest revision as of 03:11, 14 May 2026

The Jeffreys prior is a default prior distribution in Bayesian statistics named after Harold Jeffreys. It is constructed to be invariant under reparameterization — a property that makes it the natural "uninformative" prior when one wants the conclusions of inference to depend on the data, not on how the parameter was labeled. Mathematically, the Jeffreys prior is proportional to the square root of the determinant of the Fisher information matrix: p(θ) ∝ √det I(θ).\n\nThis construction reveals that the Jeffreys prior is not merely a conventional choice but a geometric one: it is the volume element of the statistical manifold equipped with the Fisher-Rao metric. What looks like a subjective Bayesian choice is, from the geometric perspective, as objective as measuring the surface area of a sphere. The prior is not an expression of belief but a measure of the intrinsic size of the parameter space.\n\nThe Jeffreys prior has limitations: it can be improper (not normalizable) for unbounded parameters, and it may perform poorly for multiparameter problems where the cross-parameter correlations in the Fisher information matrix produce counterintuitive shapes. In such cases, reference priors and other objective Bayesian constructions attempt to preserve invariance while addressing these pathologies.\n\n\n\n

@@ Line 1: / Line 1: @@
-'''Jeffreys prior''' is a method for constructing objective prior distributions in [[Bayesian statistics|Bayesian inference]], proposed by the geophysicist and statistician [[Harold Jeffreys]] in 1946. It is designed to be minimally informative in a precisely defined sense: the prior is proportional to the square root of the determinant of the [[Fisher information]] matrix, which means it assigns more probability to parameter regions where the data would be more informative.
+The '''Jeffreys prior''' is a default prior distribution in [[Bayesian Probability|Bayesian statistics]] named after [[Harold Jeffreys]]. It is constructed to be invariant under reparameterization — a property that makes it the natural "uninformative" prior when one wants the conclusions of inference to depend on the data, not on how the parameter was labeled. Mathematically, the Jeffreys prior is proportional to the square root of the determinant of the [[Fisher information]] matrix: ''p(θ) ∝ √det I(θ)''.\n\nThis construction reveals that the Jeffreys prior is not merely a conventional choice but a geometric one: it is the volume element of the [[statistical manifold]] equipped with the Fisher-Rao metric. What looks like a subjective Bayesian choice is, from the geometric perspective, as objective as measuring the surface area of a sphere. The prior is not an expression of belief but a measure of the intrinsic size of the parameter space.\n\nThe Jeffreys prior has limitations: it can be improper (not normalizable) for unbounded parameters, and it may perform poorly for multiparameter problems where the cross-parameter correlations in the Fisher information matrix produce counterintuitive shapes. In such cases, reference priors and other objective Bayesian constructions attempt to preserve invariance while addressing these pathologies.\n\n[[Category:Mathematics]]\n[[Category:Statistics]]\n[[Category:Bayesian]]
-The rationale is elegant. A prior that is uniform in one parameterization is not uniform in another — the uniform prior for a variance is not uniform for a standard deviation. Jeffreys solved this by constructing a prior that is invariant under reparameterization: if you transform the parameter, the Jeffreys prior transforms in a way that preserves its information-theoretic character. This invariance makes it a genuine default prior rather than a convenient choice.
-Jeffreys prior is not always proper — it may not integrate to one over the entire parameter space — and it can produce counterintuitive results in high dimensions, where it tends to concentrate probability in ways that favor simpler models. In [[Bayesian model selection|Bayesian model selection]], this property connects Jeffreys prior to automatic complexity penalization, blurring the boundary between prior choice and model comparison.
-The deeper significance of Jeffreys prior is that it represents an attempt to extract a unique, data-driven prior from the likelihood itself, collapsing the distinction between what we believe before seeing data and what the data model tells us about where information will be found. Whether this collapse is a methodological convenience or a philosophical confusion remains debated.
-''Jeffreys prior is the formal expression of a seductive idea: that we can derive what we should believe from the structure of what we are trying to learn. The idea is seductive because it promises to eliminate subjectivity from Bayesian inference — to make the prior 'objective' by grounding it in the mathematics of the likelihood. But the promise is hollow. The Fisher information matrix depends on the model, and the model is chosen, not discovered. Jeffreys prior is not objective; it is objective relative to a model that is itself a contingent choice. It replaces the subjectivity of prior belief with the subjectivity of model specification — and the latter is often less visible and therefore more dangerous.''
-[[Category:Mathematics]] [[Category:Statistics]]