AdaBoost: Difference between revisions

Revision as of 20:08, 14 July 2026

@@ Line 1: / Line 1: @@
-'''AdaBoost''' (Adaptive Boosting) is a sequential ensemble algorithm that trains a series of weak classifiers — typically shallow [[Decision tree|decision trees]] — and combines them into a strong classifier by weighted majority voting. The algorithm maintains a distribution over training examples that is iteratively reweighted: misclassified examples receive higher weights, forcing each subsequent learner to focus on the cases that previous learners got wrong. Introduced by Freund and Schapire in 1997, AdaBoost was the first boosting method to achieve both theoretical guarantees and practical dominance, proving that a sequence of weak learners could be combined to produce error rates arbitrarily close to zero on the training set.
-AdaBoost's elegance masks a critical vulnerability: it is extremely sensitive to noise and outliers. Mislabeled examples acquire ever-increasing weight and can dominate the final model, causing the ensemble to memorize errors rather than signal. This sensitivity was the motivation for more robust successors such as [[Gradient boosting|gradient boosting]] and its regularized variants. The algorithm's legacy is therefore dual: it established the theoretical possibility of boosting and simultaneously exposed the structural limits of exponential reweighting as a training strategy.
 [[Category:Computer Science]]
 [[Category:Artificial Intelligence]]
 [[Category:Mathematics]]