Function space

A function space is a collection of functions treated as a structured mathematical space, typically endowed with a norm, inner product, or topology that permits the application of geometric and analytic methods. In machine learning, the hypothesis space of a model is a function space — the set of all functions the model can represent — and the geometry of that space determines what the model can learn. The reproducing kernel Hilbert space (RKHS) is the paradigmatic example, providing the theoretical setting in which kernel methods and the neural tangent kernel operate. The dimensionality and spectral properties of a function space are what make benign overfitting possible or impossible: when the ambient dimension dwarfs the intrinsic dimension of the data, minimum-norm solutions can generalize despite interpolation.