Rényi entropy

Rényi entropy is a one-parameter family of entropy measures that generalizes Shannon entropy. Introduced by Alfréd Rényi in 1961, it relaxes Shannon's linear averaging requirement while preserving the core intuition that entropy measures the uncertainty of a probability distribution. For a discrete distribution with probabilities p₁, ..., pₙ and order parameter α > 0 (α ≠ 1), the Rényi entropy of order α is:

H_α(X) = (1/(1−α)) log(Σ pᵢ^α)

In the limit as α → 1, Rényi entropy converges to Shannon entropy. As α → 0, it approaches the Hartley entropy — simply the logarithm of the number of possible outcomes. As α → ∞, it approaches the negative logarithm of the maximum probability, capturing only the dominant outcome. This parametric spectrum makes Rényi entropy useful in cryptography, where it quantifies the difficulty of guessing a secret, and in ecology, where it interpolates between species richness and dominance measures.