Vector Space

A vector space (also called a linear space) is an algebraic structure consisting of a set of elements called vectors together with a field of scalars and two operations — vector addition and scalar multiplication — that satisfy a set of axioms: closure under addition and scalar multiplication, associativity, commutativity, the existence of an additive identity (the zero vector), additive inverses, and distributivity of scalar multiplication over vector addition and scalar addition.

The vector space axioms are deceptively simple. From them flows the entire edifice of linear algebra: linear combinations, spans, linear independence, bases, dimension, linear transformations, eigenvalues, and eigenvectors. The power of the vector space concept lies in its abstraction: it unifies objects that appear unrelated — arrows in the plane, sequences of numbers, functions, matrices, quantum states — under a single algebraic framework.

Euclidean and Abstract Spaces

The prototypical example is Euclidean space R^n, the set of n-tuples of real numbers with component-wise addition and scalar multiplication. The geometry of Euclidean space — length, angle, orthogonality — is encoded by the inner product, which makes R^n a Hilbert space. But the vector space axioms do not require an inner product, and many vector spaces have no natural notion of distance or angle.

Function spaces are vector spaces whose elements are functions. The space of all continuous functions on an interval, the space of all square-integrable functions, and the space of all polynomials of degree at most n are all vector spaces. In these spaces, vector addition is pointwise addition of functions, and scalar multiplication is pointwise scaling. The fact that functions can be treated as vectors is the mathematical basis of Fourier analysis, quantum mechanics, and the theory of partial differential equations.

Matrix spaces are vector spaces of matrices, with addition and scalar multiplication defined entry-wise. The space of m × n matrices over a field is isomorphic to R^(mn) as a vector space, but it carries additional structure — matrix multiplication — that makes it a ring as well as a vector space.

Vector Spaces in Systems and Information Theory

The vector space framework is indispensable in the analysis of complex systems. In dynamical systems theory, the state space of a system is a vector space (or manifold), and the system's evolution is described by a flow or map on that space. The Lyapunov exponents, which measure the rate of divergence of nearby trajectories, are defined in terms of the linearization of the dynamics — the Jacobian matrix acting on the tangent space, which is a vector space.

In information theory, the space of probability distributions over a finite set can be embedded in a vector space, and the geometry of this space — the information geometry of Fisher metrics and divergence functions — is the mathematical foundation of statistical inference and machine learning. The Hamming distance and related metrics on coding spaces are vector-space constructions: the set of all binary strings of length n is the vector space F_2^n over the finite field with two elements.

In machine learning, data representations are vectors in high-dimensional spaces, and the learning process is a search for optimal parameters in a vector space of model weights. The neural network training process — gradient descent on a loss landscape — is a vector-space operation: the gradient is a vector, the update is a vector addition, and the trajectory through parameter space is a path in a vector space.

Dimension, Basis, and the Structure of Information

The dimension of a vector space is the number of vectors in any basis — a maximal linearly independent set. Dimension is the invariant that distinguishes vector spaces: all vector spaces of the same dimension over the same field are isomorphic. This means that the complexity of a vector space is captured entirely by its dimension, not by the nature of its elements.

This fact has profound implications for information representation. If a system can be described as a vector in a finite-dimensional space, then its information content is bounded by the dimension. If the system is infinite-dimensional — a function space, a quantum state space — then its information content is potentially unbounded. The choice of basis determines which coordinates are explicit and which are implicit; a change of basis can reveal hidden structure or simplify a problem.

The eigenvalue decomposition of a linear operator is a change of basis that diagonalizes the operator, decoupling the system's dynamics into independent modes. This is the mathematical heart of principal component analysis, Fourier analysis, and the normal modes of physical systems. In every case, the vector space structure provides the language in which the decomposition is expressed.

The vector space is one of the most successful abstractions in mathematics because it captures what is common to structure across virtually every domain of science. But its very success is also a trap. When every system is treated as a vector space, the distinctive features of each system — its topology, its dynamics, its constraints — can be erased. The information geometry of probability distributions is not the same as the geometry of Euclidean space, even though both are vector spaces. The state space of a quantum system is not the same as the state space of a classical system, even though both can be described as Hilbert spaces. The vector space is a starting point, not an endpoint. The question is not whether a system can be represented as a vector space. The question is what is lost in that representation.