Out-of-bag error
Out-of-bag error is the prediction error of an ensemble model — typically a random forest or bagged ensemble — measured on the observations that each individual base learner did not see during its bootstrap training. Because each tree in a random forest is trained on roughly two-thirds of the data (drawn with replacement), the remaining one-third acts as an implicit validation set. The out-of-bag predictions are aggregated across all trees for which a given observation was left out, producing a cross-validated estimate of generalization performance without requiring a separate held-out test set. This is not a convenience trick but a statistical consequence of the bootstrap sampling procedure: the out-of-bag estimate is approximately equivalent to leave-one-out cross-validation, but computed at no additional computational cost.
The out-of-bag error is a reliable diagnostic, but it is not infallible. It tends to be pessimistically biased for small datasets and can be unstable when the number of trees is small. More importantly, it measures the ensemble's performance, not the performance of any individual tree, and it does not capture the model's behavior on data drawn from a different distribution. A model with low out-of-bag error may still fail catastrophically under distribution shift or adversarial perturbation. The error is a snapshot of internal consistency, not a guarantee of external validity.
The deeper significance of the out-of-bag error is epistemological. It represents a form of self-validation: the model tests itself using the data it has already seen, but from a different angle. This is the logic of ensemble learning at its most elegant — not merely combining predictions for accuracy, but combining them for self-awareness. The ensemble knows what it does not know because the trees that never saw a particular example can vote on it, and their disagreement is a measure of the model's uncertainty. Out-of-bag error is the closest that machine learning has come to building a system that evaluates its own limits from within.