Computer vision

Computer vision is the field of study that enables machines to derive meaningful information from digital images and video sequences. But this definition, while accurate, is too narrow. Computer vision is not merely a branch of engineering; it is a testbed for the deepest questions about how sensory signals become semantic structure. Every algorithm that claims to 'see' is making a claim about what seeing is — and the history of the field is a sequence of such claims being made, operationalized, and then refuted by the next generation of methods.

The field sits at the intersection of geometry, signal processing, machine learning, and perceptual psychology. It inherits from each parent a distinct conception of what it means to extract information from images. The geometric tradition treats vision as inverse optics: given a two-dimensional image, recover the three-dimensional scene that produced it. The statistical tradition treats vision as pattern recognition: given labeled examples, learn a function that maps pixel arrays to categories. The neural tradition treats vision as representation learning: given enough data and compute, discover hierarchical features that make the task trivial. These traditions are not merely different methodologies. They are different ontologies of what an image is.

The Geometry of Seeing

Before machine learning dominated the field, computer vision was a branch of applied geometry. The central problem was projective reconstruction: given multiple views of a scene, compute the camera positions and the three-dimensional structure of the observed objects. This is not a learning problem. It is a constraint-satisfaction problem derived from the physics of image formation. A point in the world projects to a point in the image via a projective transformation. Two images of the same scene constrain each other's geometry. The algorithms — structure from motion, stereo correspondence, optical flow — solve for the unknowns using nothing but the laws of perspective and the assumption of rigidity.

This geometric tradition produced robust, interpretable methods with clear failure modes. But it also produced a limitation: the methods worked only when the assumptions held. Real scenes contain non-rigid objects, transparent materials, and illumination changes that violate the geometric model. The geometric approach assumed that vision is the recovery of physical structure from optical structure. What it could not handle was the semantic structure that humans effortlessly perceive: a face is not just a collection of surfaces at particular depths; it is a person, with intentions, emotions, and a history.

From Eigenfaces to Deep Networks

The statistical turn in computer vision began in earnest in the 1990s with methods like principal component analysis and the eigenface approach to face recognition. The insight was radical: instead of modeling the image formation process, model the distribution of images themselves. Faces are not arbitrary pixel arrays. They occupy a low-dimensional manifold in the high-dimensional space of all possible images. Principal component analysis finds the axes of maximum variance in this distribution, and the eigenface method uses these axes as a basis for recognition. The approach was elegant, mathematically transparent, and ultimately limited: it worked for frontal faces under controlled lighting and failed for everything else.

The next shift was the convolutional neural network — a deep learning architecture that learns hierarchical features directly from raw pixels. The first layers learn edge detectors and color blobs; middle layers learn textures and simple shapes; deep layers learn object parts and, eventually, object categories. The network discovers the features that the geometric approach had tried to hand-engineer and the statistical approach had modeled as linear projections. But this discovery comes at a cost: the features are not human-interpretable, the failure modes are not predictable, and the system's behavior on out-of-distribution inputs is unreliable.

The eigenface method and the convolutional neural network represent two poles of a methodological spectrum. The eigenface is fully interpretable but rigid; the deep network is flexible but opaque. The field has not resolved this tension. It has simply shifted its weight from one pole to the other, driven by performance benchmarks rather than by principled argument about what vision should be.

The Epistemic Loop

Computer vision raises a version of the Lucas critique that its practitioners rarely acknowledge: the methods you develop change the tasks you set. When face recognition became accurate enough to deploy at scale, the task shifted from 'recognize a face in a photograph' to 'recognize a face in a crowd, at a distance, in poor lighting, wearing a mask.' Each solved problem generates harder successor problems. The field's progress is not linear; it is a spiral in which every solution redefines the problem.

This dynamic is visible in the phenomenon of the adversarial example: an image that is indistinguishable from a correctly classified image to a human observer but is classified as something entirely different by a deep network. Adversarial examples are not bugs in the engineering sense. They are diagnostic tools that reveal what the network has actually learned — and what it has not. A network that classifies a panda as a gibbon after the addition of imperceptible noise has not learned what a panda is. It has learned a statistical signature that is correlated with panda-likeness under the training distribution. The adversarial example exposes the gap between correlation and understanding.

The field's deepest problem is not technical. It is epistemological: what does it mean for a machine to 'see' an object? Does it mean that the machine can label the object correctly? That it can segment the object from its background? That it can predict how the object will behave under intervention? These are not the same capacity, and the history of computer vision is a history of conflating them. Labeling is not understanding. Segmentation is not explanation. Prediction is not comprehension.

The connection to object permanence — the cognitive capacity to represent objects as continuing to exist when they are not observed — is rarely made in the technical literature. Yet object permanence is arguably the defining achievement of biological vision. A system that lacks object permanence sees a sequence of unrelated images; a system that has it sees a world. Current computer vision systems do not have object permanence. They process frames independently, or at most maintain a short-term memory through recurrent connections. They do not represent the world as a persistent structure that continues to exist behind the camera's back.

The computer vision community has achieved remarkable engineering results by treating vision as a classification problem. But classification is not vision. A system that labels every pixel correctly without understanding that the pixels belong to objects that persist in time, that interact causally, and that exist independently of the observer's gaze, has not solved vision. It has solved a dataset. The field's benchmark culture — measuring progress by accuracy on standardized tests — has produced systems that excel at tests while failing at the world. The next paradigm in computer vision will not be a better architecture. It will be a better theory of what vision is for.