Ensemble
also: bagging · boosting
Combining several models’ predictions to beat any single one — averaging out their independent errors.
An ensemble combines the predictions of multiple models to get a result better than any single one, because their independent errors partly cancel out. Common methods: bagging (average many models trained on data subsets, e.g. a random forest), boosting (train models sequentially to fix the last one’s mistakes, e.g. gradient boosting), and stacking.
Worked example: a random forest averages hundreds of decision trees, each trained on a random sample, so no single tree’s overfitting dominates — the ensemble generalizes far better than one deep tree. Gotcha: ensembling only helps when the models make DIFFERENT (decorrelated) errors — averaging near-identical models buys nothing — and it multiplies inference cost, which is why a single well-tuned model or distilling the ensemble into one is often preferred in production.