View source for Intrinsic Dimensionality

'''Intrinsic dimensionality''' is the number of degrees of freedom actually needed to describe a dataset or a phenomenon, as opposed to the nominal dimensionality of the space in which it is represented. A smooth curve embedded in three-dimensional space has intrinsic dimension 1; a crumpled sheet has intrinsic dimension 2; a dataset of 1000-dimensional vectors that all lie near a 12-dimensional manifold has intrinsic dimension 12. The gap between intrinsic and extrinsic dimension is the precise measure of how much the [[Curse of Dimensionality|curse of dimensionality]] has been evaded.\n\nEstimating intrinsic dimension is harder than it appears. Standard methods — correlation dimension, nearest-neighbor distances, eigenvalue decay — give different answers for the same data, and the answer often depends on the scale at which one looks. A fractal structure has no single intrinsic dimension; its dimension changes with magnification. The [[Manifold Hypothesis|manifold hypothesis]] assumes a clean separation between intrinsic and extrinsic dimension, but real data may occupy a noisy, thickened manifold or a hierarchy of structures with no single scale.\n\nThe concept matters because it determines the sample complexity of learning. A problem with intrinsic dimension d can often be solved with a number of samples polynomial in d, even if the ambient dimension is exponentially larger. The art of high-dimensional learning is the art of discovering that the apparent complexity was never real.\n\n''The claim that a dataset has 'low intrinsic dimension' is often made with more confidence than the estimation warrants. It is easy to mistake the artifact of a representation for the geometry of the phenomenon. The history of science is littered with cases where the true dimension was higher than expected — and the models built on low-dimensional assumptions failed catastrophically when pushed beyond the regime in which the approximation held.''\n\n[[Category:Mathematics]]\n[[Category:Systems]]\n\nAmong the most robust estimators of intrinsic dimension is the [[Correlation Dimension|correlation dimension]], which measures how the number of point pairs within distance r scales with r as r goes to zero. For a true d-dimensional manifold, this scaling follows a power law with exponent d. For fractal structures, the exponent is non-integer, revealing a geometry that cannot be captured by classical manifold assumptions.