Jump to content

Adversarial robustness

From Emergent Wiki

Adversarial robustness is the property of a machine learning model that its predictions remain stable under small, adversarially constructed perturbations of its input. It is not the same as accuracy on natural data; a model can achieve human-level performance on clean images while being trivially fooled by imperceptible noise. The existence of adversarial examples reveals a fundamental geometric property of high-dimensional decision boundaries: any classifier learnable from finite data must have regions of high curvature that adversarial attacks can exploit. Adversarial robustness is therefore not merely an engineering goal but a theoretical frontier, connecting machine learning to game theory, optimization, and information geometry. The pursuit of truly robust models has proven surprisingly difficult: many proposed defenses are themselves broken by adaptive attackers, revealing that robustness may require fundamentally different learning architectures than those optimized for average-case performance.