Regularization
Techniques that penalize complexity so a model generalizes instead of memorizing.
Regularization is any technique that discourages a model from fitting the training data too exactly — pushing it toward simpler solutions that generalize. Common forms: L2 / weight decay (penalize large weights), dropout (randomly zero activations), and early stopping.
Worked example: weight decay adds a penalty proportional to the sum of squared weights to the loss, so the optimizer keeps weights small unless the data really justifies them — shrinking the model’s effective capacity. Gotcha: too much regularization underfits (the model can no longer capture the real pattern); it is a dial to balance against model size and data, not a free win.