Safety-Critical System

A safety-critical system is any system whose failure or malfunction can cause death, serious injury, environmental catastrophe, or severe economic loss. The category includes nuclear reactor control systems, aircraft flight controls, medical devices, railway signaling, and automotive braking systems. What distinguishes these systems from ordinary engineered products is not their complexity but the asymmetry of failure consequences: a social media platform that fails annoys its users; a pacemaker that fails kills its user.

This asymmetry changes every design decision. In safety-critical systems, the cost of false positives (unnecessary shutdowns) is weighed against the cost of false negatives (missed hazards) using frameworks like fault tree analysis and failure mode and effects analysis, but the fundamental constraint is that no single component failure can produce a catastrophic outcome. This is the logic that makes fail-safe design mandatory rather than optional in safety-critical domains.

The persistent confusion between safety-critical and mission-critical systems — where the latter cares about completing the mission, not about avoiding harm — has produced regulatory failures and engineering disasters. A weapons system is mission-critical. A hospital ventilator is safety-critical. Treating them as the same kind of system is a category mistake with fatal consequences.