Validation
Validation is the process of establishing that a model, system, or claim corresponds to the reality it purports to represent — a process distinct from verification, which checks internal consistency against a specification. Where verification asks 'did we build the thing right?' validation asks 'did we build the right thing?' This distinction, introduced by Barry Boehm in software engineering, generalizes far beyond software: any system that maps an abstract representation onto a concrete domain must confront the validation problem, and the problem is never fully solvable.
The validation challenge is epistemological at its core. A model is always a simplification, and the simplification discards information that may turn out to be relevant. A climate model validated against historical temperature records may fail to predict a future tipping point because the validation data did not contain the tipping dynamics. A machine learning model validated on a training distribution may fail in deployment because the deployment environment is not sampled from the same distribution. Validation is not a binary state — validated or not validated — but a continuous process of testing the boundaries of a model's applicability.
Validation in Complex Systems
In complex systems, validation becomes a network problem rather than a point problem. A single component can be verified against its specification, but the system's behavior emerges from interactions that no component-level specification captures. A distributed system may pass every unit test and every integration test and still fail in production because the interaction topology in production differs from the test environment. The emergent properties of the system — latency cascades, deadlock patterns, consensus failures — are not properties of any individual component, and they cannot be validated by validating the components.
This is the validation gap: the space between what can be tested in isolation and what actually happens in the integrated system. The gap is not a failure of testing methodology. It is a structural feature of any system whose behavior is not compositional — any system where the whole is not merely the sum of its parts. In such systems, validation requires not just more tests but different tests: stress tests that probe the interaction space, simulations that explore counterfactual configurations, and operational monitoring that treats the running system as a continuously validated experiment.
The epistemic capture problem is a special case of the validation gap. When a system's validation regime becomes too narrow — when it tests only the scenarios that the designers anticipated — the system appears validated while remaining vulnerable to unanticipated failure modes. The validation itself becomes a source of blind spots, creating the illusion of safety where fragility exists. This is not a technical failure but a cognitive one: the validation regime validates the modeler's assumptions rather than the system's reality.
The Social Dimension of Validation
Validation is not purely technical. It is a social process that depends on trust, authority, and institutional context. A clinical trial validates a drug not through abstract epistemology but through a regulated process of peer review, statistical significance testing, and regulatory approval. The validation is credible not because the methodology is perfect but because the process is transparent, reproducible, and subject to institutional oversight. When these institutional conditions fail — when peer review is corrupted, when statistical standards are gamed, when regulatory capture occurs — the validation loses its epistemic authority.
The replication crisis in psychology and other social sciences is a validation crisis. Experiments that were statistically validated in original studies failed to replicate when conducted by independent researchers. The failure was not necessarily fraud or incompetence. It was a structural feature of a validation regime that rewarded novel, surprising findings over reproducible, incremental ones. The validation system was optimized for publication, not for truth, and the optimization produced a distorted map of the underlying reality.
Validation is not a destination. It is a process of continuous approximation, and the approximation is always incomplete. The belief that a system is validated is itself a cognitive state that requires validation: under what conditions does the belief remain justified? The answer is that validation is a social and technical practice that must be maintained, not a property that can be achieved. A validated system is a system whose validation is still working. When the validation stops, the system reverts to the status of an untested hypothesis, regardless of how many tests it once passed.