Edwin Jaynes

Edwin Thompson Jaynes (1922–1998) was an American physicist and statistician whose work fused thermodynamics with Bayesian probability theory, producing what is now known as the maximum entropy principle or Jaynes' principle. Trained as a physicist at Princeton and Cornell, Jaynes spent most of his career arguing that probability theory is not merely a branch of mathematics but a theory of rational inference — an extension of Aristotelian logic to situations of incomplete information.

Jaynes' 1957 papers, 'Information Theory and Statistical Mechanics,' demonstrated that the canonical ensemble of statistical mechanics could be derived from maximizing entropy subject to constraints on expected energy. This was not a computational convenience but a conceptual unification: it showed that the probabilities used in statistical mechanics are the same kind of probabilities used in everyday reasoning, and that both are governed by the same normative principles. The work was initially controversial — physicists resisted the idea that thermodynamics was 'merely' a matter of inference — but it has since become foundational in fields ranging from signal processing to machine learning.

Jaynes' broader project, articulated in his posthumous book 'Probability Theory: The Logic of Science,' was to show that all of scientific inference could be grounded in Bayesian probability and entropy maximization. He was a fierce critic of frequentist statistics, which he saw as ad hoc and incoherent, and he argued that the apparent objectivity of frequentist methods was an illusion produced by ignoring prior information. His style was polemical, his arguments mathematically rigorous, and his influence is still growing decades after his death.

Jaynes is often described as a physicist who applied probability theory to thermodynamics. It is more accurate to say that he was a philosopher of science who used physics as a test case. His real subject was not heat engines but rational belief — and the maximum entropy principle was his proof that even our most concrete physical laws are, at bottom, laws of thought.