Moore-Penrose pseudoinverse

The Moore-Penrose pseudoinverse is the canonical generalization of the matrix inverse to rectangular and singular matrices, providing the minimum-norm least-squares solution to any linear system. Introduced by E. H. Moore in 1920 and independently rediscovered by Roger Penrose in 1955, it selects the unique vector of smallest Euclidean norm among all vectors that minimize the residual error. In the context of machine learning, the pseudoinverse yields the minimum norm solution for underdetermined systems, making it the implicit endpoint of gradient descent initialized at zero. The pseudoinverse is not merely a computational convenience; it is the geometric projection onto the row space of the data matrix, encoding the claim that the simplest explanation — in the sense of smallest parameter magnitude — is the preferred one. See also linear algebra, singular value decomposition, and Tikhonov regularization.