Canonical correlation analysis

Canonical correlation analysis (CCA) is a multivariate statistical method developed by Harold Hotelling in 1936 that finds the linear relationships between two sets of variables. Where principal component analysis finds the directions of maximum variance within a single dataset, CCA finds the directions of maximum correlation between two datasets. The method computes pairs of canonical variates — linear combinations of variables from each set — such that the correlation between the first pair is the highest possible, the correlation between the second pair is the highest possible subject to being uncorrelated with the first, and so on. This is the natural generalization of the Pearson correlation coefficient to the multivariate case: instead of correlating two scalar variables, CCA correlates two vector spaces.

CCA has found applications across domains that its inventor could not have anticipated. In neuroscience, it is used to find correspondences between brain activity and behavioral variables. In genomics, it identifies relationships between gene expression and clinical phenotypes. In natural language processing, it serves as a foundational technique for aligning semantic spaces across languages — the geometric intuition that two vector spaces can be rotated onto each other so that their axes of maximum correlation coincide. The method assumes linear relationships and Gaussian distributions, assumptions that are increasingly replaced by deep neural network alternatives. Yet CCA remains the clearest example of a purely geometric approach to measuring association between complex systems.

CCA is the forgotten sibling of PCA. Where PCA dominates the textbooks, CCA languishes in specialized chapters — yet the question CCA answers is arguably more important. We rarely care about the internal structure of a single dataset in isolation. We care about how two systems relate to each other. CCA is the linear algebra of relationship, and its relative neglect says more about the individualistic bias of statistical pedagogy than about the method's limitations.