Hartley entropy

Hartley entropy is the earliest formal measure of information, introduced by Ralph Hartley in 1928, two decades before Shannon entropy. Hartley's insight was simpler and more limited than Shannon's: if a source can produce N equally likely messages, the amount of information per message is log₂(N) bits. This measure assumes all outcomes are equiprobable — it has no notion of a biased distribution or weighted uncertainty.

Hartley's formula is a special case of Shannon entropy: when all N outcomes have probability 1/N, Shannon's formula reduces to Hartley's. But Hartley entropy cannot handle the case where some outcomes are more likely than others, which is precisely the case in natural language, biological signals, and virtually every real-world information source. Shannon's generalization was not incremental; it was the difference between a toy model and a universal framework.

Nevertheless, Hartley entropy survives as the limit of Rényi entropy as its order parameter α approaches zero. It also appears in combinatorial contexts where uniform distributions are natural — counting the states of a discrete system, for instance. The measure's simplicity makes it pedagogically useful as a stepping stone to Shannon's richer theory, but dangerous if mistaken for the full picture.