Overdispersion

Overdispersion occurs when the observed variance in a dataset exceeds the variance predicted by a statistical model — most commonly, when the variance of count data exceeds its mean, violating the equidispersion assumption of the Poisson distribution. It is the signature of unmodeled heterogeneity: individuals, events, or units that appear identical to the model are in fact different in ways that matter for the outcome.

In epidemiology, overdispersion is not a statistical nuisance. It is the empirical fingerprint of superspreading. When the variance of secondary infections far exceeds the mean, the Poisson assumption of the classical SIR model fails, and the epidemic is driven by rare, high-leverage transmission events rather than by average behavior. The negative binomial distribution replaces the Poisson precisely to model this overdispersion, introducing a dispersion parameter k that captures the degree of heterogeneity.

Overdispersion appears whenever a model assumes homogeneity that does not exist. In ecology, spatial heterogeneity produces overdispersed species counts. In genetics, unmeasured population structure produces overdispersed allele frequencies. In finance, correlated defaults produce overdispersed loss distributions. In every case, overdispersion is a signal that the model has missed a source of variation — and that the missing source is often the most important one.

Overdispersion is the statistical profession's way of admitting that averages lie. The Poisson distribution assumes a world in which everyone is the same. The negative binomial admits a world in which a few individuals matter more than all the others. The second world is the real one.