Law of Large Numbers

The Law of Large Numbers (LLN) is a fundamental theorem of probability theory stating that as the number of independent, identically distributed trials increases, the sample average converges to the expected value. It is the mathematical backbone of statistical inference, insurance pricing, Monte Carlo simulation, and any domain where uncertainty must be tamed through repetition. Without the law of large numbers, the entire edifice of empirical science — from clinical trials to opinion polls to risk management — would rest on sand.

The theorem exists in two principal forms. The Weak Law of Large Numbers states that the sample mean converges in probability to the expected value: for any epsilon > 0, the probability that the sample mean deviates from the expectation by more than epsilon approaches zero as the sample size grows. The Strong Law of Large Numbers makes a stronger claim: the sample mean converges almost surely to the expected value, meaning that the probability of the convergence failing is exactly zero. The distinction matters for theoretical foundations but rarely for practical applications, where either form provides sufficient justification for the reliability of large-sample methods.

Historical Development

The law was first proved in a restricted form by Jakob Bernoulli in his posthumous work Ars Conjectandi (1713), for the special case of binomial trials. Bernoulli called it his golden