Gradient boosting builds an ensemble of trees sequentially, where each new tree is trained to correct the residual errors of the trees so far. It adds weak learners one at a time in the direction that most reduces the loss — libraries like XGBoost, LightGBM, and CatBoost made it the dominant approach for tabular data and Kaggle competitions.
Worked example: the first tree makes rough predictions; the second learns to predict the first’s errors; the third corrects the remaining error, and so on — the sum converges to a strong model. Gotcha: because it fits errors sequentially, gradient boosting can overfit if you add too many trees or too high a learning rate — unlike random forests, where more trees only help — so it needs early stopping and careful tuning of tree count, depth, and learning rate.