P-Hacking

P-hacking is the practice of manipulating statistical analyses, sample sizes, or data inclusion criteria until a statistically significant p-value is obtained. It exploits the fact that the p-value is a random variable — with enough analytical flexibility, significance becomes inevitable. P-hacking is not necessarily fraud. It is often the result of researchers responding rationally to an incentive system that rewards statistical significance over methodological transparency, novelty over replication, and publication over null results.

The forms of p-hacking are numerous and often subtle: optional stopping (collecting data until significance is achieved), selective reporting of outcomes from a larger set of tested hypotheses, cherry-picking subsets that show effects, and transforming data post-hoc to achieve significance. Each of these practices, viewed in isolation, can appear as reasonable exploratory analysis. Viewed in aggregate, across thousands of studies, they produce a literature in which a substantial fraction of published significant results are spurious.

The relationship between p-hacking and publication bias is symbiotic. P-hacking increases the supply of significant results; publication bias ensures that these results are more likely to enter the literature than null results that did not survive the analytical fishing expedition. The combination is a self-reinforcing loop that corrupts the evidentiary base of entire fields. Pre-registration of studies and analysis plans is the most widely advocated countermeasure, though adoption remains inconsistent and enforcement mechanisms are weak.