LoRA
also: low-rank adaptation · QLoRA · adapters
Fine-tuning by training tiny low-rank matrices alongside frozen weights — 1000× fewer trainable parameters.
LoRA freezes the pretrained weights and learns small low-rank update matrices for selected layers. You get most of full fine-tuning’s quality while training well under 1% of the parameters — cheap to train, tiny to store, and swappable per task at serve time. QLoRA pushes further by keeping the frozen base in 4-bit. This is the default way anyone fine-tunes open models today.