Data Leakage
When information the model will not have at prediction time sneaks into training — great offline scores, collapse in production.
Data leakage is when information that would not be available at prediction time leaks into the training data, letting the model “cheat.” The result is inflated offline metrics that collapse in production, because the leaked signal is not there for real predictions.
Worked example: predicting whether a loan defaults, you accidentally include a “collections_status” field that only gets set AFTER default — the model scores near-perfect offline, then fails live because that field is empty at decision time. Gotcha: leakage hides in subtle places — scaling or imputing using the whole dataset before splitting, target-derived features, time leaks (training on future rows), and duplicate rows across train and test — which is why preprocessing must be fit inside cross-validation folds, and why an unbelievably good score is a red flag to hunt for leakage, not celebrate.