Effect Size

Effect size is a quantitative measure of the magnitude of a phenomenon, independent of sample size and statistical significance. Where a p-value answers the question is this effect distinguishable from zero?, an effect size answers how large is this effect? — a different question with different inferential implications. Common measures include Cohen's d (standardized mean difference), Pearson's r (correlation coefficient), and eta-squared (proportion of variance explained).

The effect size movement emerged as a response to the replication crisis, which exposed that statistically significant results in small samples often correspond to trivially small effects that fail to replicate. A study with p = 0.01 and Cohen's d = 0.1 has found a "significant" result that is practically meaningless. Effect sizes force researchers to confront the magnitude of what they have found, not merely its existence.

From a systems perspective, effect size is a measure of signal strength in noise, and its interpretation depends on the context of the system being studied. In physics, a small effect size may be revolutionary; in education, a moderate effect size may be transformative at scale. The demand for universal thresholds — Cohen's conventions of small (0.2), medium (0.5), and large (0.8) — is itself a systems error: it treats all domains as comparable when their baselines, variances, and feedback structures differ fundamentally.