Kolmogorov-Sinai Entropy

Kolmogorov-Sinai entropy is the intrinsic rate at which a dynamical system generates information. Defined by Andrey Kolmogorov in 1958 and refined by Yakov Sinai in 1959, it measures the asymptotic entropy per unit time of a system's trajectory, maximised over all possible finite partitions of phase space. A system with positive Kolmogorov-Sinai entropy is, by definition, chaotic: it amplifies microscopic uncertainty into macroscopic unpredictability at an exponential rate.

Formally, for a measure-preserving dynamical system (X, μ, T) and a finite partition P of X, the entropy of the partition refines as the system evolves. The Kolmogorov-Sinai entropy is the supremum over all finite partitions:

h_KS = sup_P h(T, P)

where h(T, P) is the entropy rate of the symbolic dynamics induced by P. The supremum ensures that h_KS captures the system's intrinsic information production, independent of how an observer chooses to coarse-grain the phase space.

The Kolmogorov-Sinai entropy connects three seemingly separate domains: the thermodynamic entropy production of statistical mechanics, the Shannon entropy of information theory, and the block entropy of symbolic dynamics. That these three quantities converge on the same mathematical object is either the deepest structural fact about physical information or a coincidence we have not yet learned to see past. The conflation of thermodynamic and information-theoretic entropy remains contested — but the Kolmogorov-Sinai entropy is where the mathematics itself refuses to choose between them.