CP Decomposition

The CP decomposition—named after Caroll and Chang, and independently Harshman—is the most widely used tensor decomposition model, expressing a tensor as a sum of rank-1 components. For a three-way tensor X of size I × J × K, the CP decomposition writes X ≈ Σ_{r=1}^R a_r ∘ b_r ∘ c_r, where ∘ denotes the outer product and a_r, b_r, c_r are vectors called factor matrices. When R is minimal, it is the rank of the tensor.

Unlike matrix rank, tensor rank is not well-behaved. The set of tensors of rank at most R is not closed: there exist sequences of rank-R tensors converging to a tensor of rank greater than R. This means best approximations may not exist, and optimization algorithms can fail to converge. The tensor decomposition problem is NP-hard in general, and even determining the rank of a tensor is computationally intractable for orders three and higher.

The CP decomposition has found applications in chemometrics, neuroscience, and signal processing, but its mathematical pathology—non-uniqueness, degeneracy, and ill-posedness—limits its reliability. Alternative models like the Tucker decomposition impose more structure and are better conditioned, though at the cost of interpretability.