Robustness (machine learning)

Robustness in machine learning is the property of a model that its predictions remain stable under small perturbations of the input. A robust image classifier classifies a cat as a cat even when the image is slightly rotated, blurred, or compressed. A non-robust classifier may change its prediction dramatically under perturbations that a human would not even notice. The study of robustness has become central to machine learning because modern deep networks are, by default, not robust.

The canonical example of non-robustness is the adversarial example: a carefully constructed perturbation that is invisible to human perception but causes a network to misclassify with high confidence. Adversarial examples reveal that the network's decision surface is not smooth in the way that human perception is smooth. The network has learned statistical correlations that are fragile — they hold in the training distribution but break in directions that carry no semantic meaning for humans. This is not a failure of engineering. It is a consequence of the network's objective: to minimize classification error on a finite dataset, not to learn robust, human-like concepts.

The robustness literature has produced two main approaches. The first is adversarial training: training the network on adversarially perturbed examples so that it learns to classify correctly even under attack. This improves robustness but typically reduces accuracy on clean examples. The trade-off between robustness and accuracy is not an engineering problem to be solved. It is a fundamental tension: a model that is robust to all possible perturbations must be conservative in its predictions, which means it will miss subtle but genuine patterns in the data. The second approach is certified robustness: proving, for specific model classes, that no perturbation within a given radius can change the prediction. This provides guarantees but is computationally expensive and applies only to restricted model classes.

The deeper question is whether robustness is the right goal. A model that is robust to all perturbations is a model that has learned the invariant structure of the domain — the properties that are genuinely constitutive of the categories rather than accidentally correlated with them. But in complex domains like vision and language, it is not clear what the invariant structure is. Human robustness is not achieved by formal proof. It is achieved by embodied interaction with a physical world that provides feedback about what matters. A machine learning model trained on static datasets lacks this feedback loop. Its robustness is always parasitic on the robustness of the data distribution, which is chosen by humans and reflects human biases.