Generalized Linear Model

A generalized linear model (GLM) is a flexible generalization of ordinary linear regression that allows the response variable to have an error distribution other than the normal distribution. Introduced by Ronald Fisher's successors — particularly Nelder and Wedderburn in 1972 — the GLM unifies a wide range of statistical techniques under a single framework: linear regression, logistic regression, Poisson regression, gamma regression, and survival analysis are all special cases of the GLM family.

The framework consists of three components: a random component specifying the response distribution (normal, binomial, Poisson, gamma), a systematic component specifying the linear predictor (a weighted sum of covariates), and a link function connecting the expected value of the response to the linear predictor. The link function is the GLM's signature innovation: it allows the model to handle responses that are bounded, discrete, or skewed, while preserving the linear structure of the predictor.

From a systems perspective, the GLM is a modeling grammar rather than a single model. It provides a language for specifying how a system's outputs depend on its inputs, with constraints on the output's distributional family. The grammar is powerful but limited: it assumes that the systematic and random components are separable, that observations are conditionally independent, and that the link function is correctly specified. When these assumptions fail — as they do in networked, temporal, or hierarchical data — the GLM requires extension to mixed-effects models, generalized additive models, or Bayesian hierarchical frameworks.

The GLM's historical importance is that it ended the balkanization of statistical practice. Before the GLM, each response type had its own theory, its own estimation methods, its own diagnostic tools. The GLM showed that these were instances of a single inferential architecture, distinguished only by the choice of distribution and link. This unification is analogous to the unification achieved in physics by gauge theory: different forces, same underlying symmetry principle. In statistics, different models, same underlying exponential family.