Feature Map

A feature map is a function φ: X → H that embeds raw data into a (typically high- or infinite-dimensional) Hilbert space where linear methods become powerful. In machine learning, the feature map transforms nonlinear problems in the input space into linear problems in the feature space: a dataset that is not linearly separable in X may become separable in H.

The power of the feature map is that it need not be computed explicitly. The kernel trick in reproducing kernel Hilbert spaces computes inner products in H using only the kernel function k(x, y) = ⟨φ(x), φ(y)⟩, bypassing the construction of φ entirely. This is not merely computational convenience; it is the observation that the geometry of the feature space is fully determined by pairwise similarities.

Feature maps appear throughout representation learning: in neural networks, where hidden layers learn hierarchical feature maps; in kernel methods, where the map is implicit; and in manifold learning, where the goal is to discover the low-dimensional structure that the high-dimensional feature map has captured.