Quantization
also: int8 · int4 · FP8
Storing model weights in fewer bits — smaller, faster, cheaper, with a small quality haircut.
Quantization represents weights (and sometimes activations) with lower-precision numbers — 8-bit or 4-bit integers instead of 16-bit floats. A 4-bit model needs ~4× less memory than FP16, fits on smaller GPUs, and runs faster because inference is memory-bandwidth-bound. Done well (e.g. GPTQ/AWQ-style post-training quantization), the quality drop is small; done crudely, reasoning degrades first.