Random Forest
An ensemble of many decision trees trained on random data/feature subsets, averaged — accurate and hard to overfit.
A random forest is an ensemble of many decision trees, each trained on a random subset of the data (bagging) and a random subset of features at each split. Their predictions are averaged or voted, which cancels out individual trees’ overfitting and yields a robust, accurate model with little tuning.
Worked example: 500 trees each see a different bootstrap sample and random feature subset, so they make different errors; averaging them gives a prediction far more stable than any single tree, and the feature-usage stats give a built-in importance ranking. Gotcha: random forests are a strong, low-effort baseline for tabular data (often beating neural nets there), but they lose the single tree’s interpretability and extrapolate poorly outside the training range — they predict within the box they were shown, not beyond it.