LASSO

LASSO (Least Absolute Shrinkage and Selection Operator) is a regularized regression method introduced by Tibshirani (1996) that imposes an L1 penalty on regression coefficients, driving irrelevant coefficients to exactly zero. Unlike ridge regression, which shrinks all coefficients proportionally, LASSO performs simultaneous estimation and variable selection: the resulting model is sparse, using only a subset of the available predictors.

The L1 penalty is not merely a mathematical curiosity. It corresponds to a Laplace prior over coefficients — an explicit belief that most predictors contribute zero signal and a few contribute strong signal. Whether this belief is warranted depends on the domain. In genomics, where a few causal variants drive most of the trait variance, LASSO works well. In economics, where effects are typically diffuse and highly correlated, LASSO tends to select arbitrarily among correlated predictors and miss dense signals entirely.

The central limitation of LASSO is its assumption of an approximately sparse world. This assumption fails in precisely the domains — neuroscience, social science, ecology — where researchers most want a magic variable-selection procedure. In high-dimensional regimes with dense signals, ridge regression or overparameterized models without explicit sparsity constraints typically outperform LASSO on prediction while being less interpretable. The appeal of LASSO's interpretable sparse outputs must be weighed against the systematic bias introduced when sparsity is the wrong model of reality. See also: Causal Inference, Regularization Theory, Model Selection.