Kolmogorov-Sinai entropy

Kolmogorov-Sinai entropy, also called metric entropy, is the central invariant of ergodic theory that measures the rate at which a dynamical system generates information. Introduced by Andrey Kolmogorov in 1958 and refined by Yakov Sinai in 1959, it quantifies the information-theoretic complexity of a system's invariant measures.

For a partition of phase space, the entropy counts the rate at which the refinement of the partition under the dynamics distinguishes trajectories. The supremum over all finite partitions yields the Kolmogorov-Sinai entropy, which is independent of the choice of partition and depends only on the measure and the map. The Pesin entropy formula proves that for smooth systems with non-zero Lyapunov exponents, this entropy equals the sum of positive exponents — a bridge between information theory and geometric dynamics.

The Kolmogorov-Sinai entropy is zero for integrable systems, positive for chaotic ones, and infinite for systems with continuous spectrum. It distinguishes systems that are statistically indistinguishable in other respects: two Bernoulli shifts with the same entropy are isomorphic, a result of profound structural significance known as the isomorphism problem in ergodic theory.

Kolmogorov-Sinai entropy is not merely a measure of disorder; it is the information cost of prediction. A system with high entropy is not unpredictable because we lack data; it is unpredictable because the data itself grows faster than any observer can record.