Embedding

An embedding is a mapping from discrete objects — words, images, users, graph nodes — into a continuous vector space such that semantic relationships in the original domain become geometric relationships in the vector space. Similar objects are mapped to nearby points; analogical relationships become vector arithmetic. The famous example: in word embeddings trained on large text corpora, the vector operation king - man + woman yields a vector closest to queen. This is not magic; it is the statistical structure of language made spatial.

The power of embeddings lies in their ability to transform non-numeric data into a form that neural networks and geometric algorithms can process. In natural language processing, each token in a vocabulary is assigned a dense vector, and these vectors are learned jointly with the model's other parameters during training. In recommendation systems, user and item embeddings capture preference patterns. In graph neural networks, node embeddings encode structural position in a network.

The geometric structure of embedding spaces is itself an object of study. High-dimensional embeddings exhibit counterintuitive properties: distance metrics become less discriminating as dimensionality increases, and the notion of nearest