Survivorship Bias

Survivorship bias is the logical error of concentrating attention on entities that have 'survived' a selection process while overlooking those that did not, typically because the non-survivors are invisible or absent from the data. The bias leads to systematically false conclusions about the properties that lead to success, the frequency of success, or the representativeness of observed outcomes. The canonical illustration is Abraham Wald's World War II analysis of bullet holes in returning aircraft: engineers proposed reinforcing the areas with the most damage, but Wald recognized that the aircraft in the sample were those that had survived — the planes shot down had been hit in different places. The bias appears at all scales, from individual anecdotes ('my grandfather smoked and lived to ninety') to evolutionary narratives (organisms alive today were all fit enough to survive — we cannot sample the extinct) to financial analysis (mutual funds that close are excluded from performance databases). In the life sciences, survivorship bias is particularly insidious in ecological research (we study species that survived bottlenecks, not those that went extinct), clinical medicine (patients who make it to referral centers differ systematically from those who do not), and in the study of adaptive cognition (behaviors we label 'adaptive' are selected from a biased sample of observable outcomes). Correcting for survivorship bias requires explicit modeling of the selection process, inclusion of censored or missing observations, and disciplined resistance to narrative constructions that flow too easily from the visible data. See also: Confirmation Bias, Base Rate Neglect.