Interpolation threshold

Interpolation threshold is the critical point in the capacity-data plane where a model acquires exactly enough parameters to fit its training data perfectly — to interpolate every point. In the underparameterized regime below the threshold, no model in the class can achieve zero training error. At the threshold, the set of interpolating solutions explodes from empty to infinite-dimensional, and the problem's geometry changes discontinuously. This threshold is not merely a numerical coincidence. It is the phase boundary between two regimes of learning with radically different generalization behavior: the classical U-shaped bias-variance tradeoff below, and the second descent of double descent above.

The threshold's location depends on the effective number of parameters relative to the number of training examples, but the relationship is not straightforward. Regularization, data structure, and optimization dynamics all shift the threshold. A model with implicit constraints may reach effective interpolation at a much higher nominal capacity than an unconstrained one. The threshold is where regularization transitions from explicit to implicit: below it, penalties constrain the hypothesis space; above it, the optimizer's trajectory among infinite solutions becomes the constraining force.

The interpolation threshold is the knife-edge where statistical learning theory comes apart. On one side, the classical framework works. On the other, it fails. The fact that modern machine learning operates almost exclusively on the far side of this edge is not a footnote — it is the central fact that any general theory of learning must now accommodate.