Gradient Clipping
Capping the size of the gradient before applying it, so one unusually large batch can’t blow up training.
Gradient clipping rescales the gradient if its norm exceeds a threshold, capping how large a single weight update can be. Without it, a rare batch or an unstable region of the loss landscape can produce an enormous gradient that destabilizes training for many steps afterward — clipping is cheap insurance that’s standard in nearly every large-scale training run.