Meta-analysis

Meta-analysis is the statistical synthesis of results from multiple independent studies to estimate an effect size, test a hypothesis, or resolve apparent contradictions in the literature. Rather than treating each study as an isolated observation, meta-analysis treats the literature itself as a noisy measurement apparatus and applies formal methods to extract the signal from the aggregate. The technique is central to evidence-based medicine, where individual randomized controlled trials may be underpowered, biased by funding sources, or conducted on unrepresentative populations.

But meta-analysis is not a neutral aggregator. It is a selection machine: the studies included are a non-random sample of studies conducted, distorted by publication bias (positive results are more likely to be published), file-drawer effects (null results remain unpublished), and the adverse selection of researchers toward methods that produce publishable findings. A meta-analysis of published literature is therefore a meta-analysis of a selected subset of reality, and the selection mechanism is rarely modeled in the statistical synthesis itself.

The tension is structural: meta-analysis promises certainty through aggregation, but the aggregation itself operates on data that has already been filtered by the institutional incentives of science. The resulting confidence intervals are precise about statistical uncertainty while remaining silent about selection uncertainty — the uncertainty introduced by the fact that we do not know what studies were never conducted, never submitted, or never accepted. Meta-analysis without publication bias correction is not evidence synthesis. It is evidence curation dressed in statistical formalism.