“Bagging Predictors” by Leo Breiman was published in the journal Machine Learning, volume 24, in 1996. It introduced bagging, short for “bootstrap aggregating,” a simple and general method for making predictive models more accurate and stable.
The idea is to train many copies of a model on slightly different versions of the data. Each version is a bootstrap sample, drawn at random with replacement from the original training set, so it contains some examples multiple times and omits others. The individual models are then combined, by voting for classification or averaging for regression. Because the errors of the individual models are partly independent, averaging them cancels out much of the random variation, reducing the variance of the prediction without increasing its bias.
Bagging works best with “unstable” learners, like decision trees, whose structure changes a lot when the data changes slightly. Breiman’s analysis of why this helps fed directly into his later and even more influential random forests, which add bagging to randomized trees.
Why business readers should care: bagging is one of the cheapest ways to make a model more reliable - train several on resampled data and average them - and it underlies the ensemble methods that power a large share of production machine learning.