No free lunch theorem

The no free lunch theorem in machine learning and optimization states that no algorithm dominates all others across all possible problem instances. Every algorithm that performs well on one class of problems must perform poorly on another. The theorem is not a consolation prize for mediocre methods; it is a structural claim about the geometry of problem spaces. It means that the only way to outperform a naive algorithm is to make assumptions about the structure of the problems you actually face — what is called inductive bias. The no free lunch theorem is therefore not a barrier to good performance. It is the reason that learning is possible at all: without it, no specialization could ever be justified. The theorem tells us that general-purpose intelligence is either impossible or infinitely expensive, and that every practical learner is a bet on the structure of its environment.