Pruning
also: model pruning
Removing unimportant weights or neurons from a trained model to shrink and speed it up with little accuracy loss.
Pruning compresses a trained neural network by removing parameters that contribute little — zeroing small-magnitude weights (unstructured) or dropping whole neurons, channels, or heads (structured). The goal is a smaller, faster model that keeps most of the accuracy.
Worked example: prune the 30% smallest-magnitude weights, then fine-tune briefly to recover the small accuracy drop — the result is sparser and cheaper to store, and (with structured pruning) faster to run. Gotcha: unstructured pruning produces sparse weights that most hardware cannot actually run faster without special support, so the storage win does not automatically become a speed win — structured pruning (removing whole units) is what reliably speeds up inference; pruning pairs with quantization and distillation as the compression toolkit.