QLoRA,
explained.

Training memory isn't just weights. Full fine-tuning with AdamW holds the 16-bit weights, 16-bit gradients, and two 32-bit optimizer states — roughly 16 bytes per parameter before activations. For a 65B model that's north of a terabyte; even LoRA, which shrinks gradients and optimizer state to almost nothing, still needs the frozen base model resident in 16-bit: 130 GB for 65B. The adapters were cheap; the statue they wrapped around was not.

The obvious move — store the frozen base in 4-bit — had an obvious objection: everyone "knew" 4-bit was for inference, and training through quantized weights would degrade everything. QLoRA's contribution is showing that with the right data type and plumbing, the objection is wrong.

QLoRA freezes the base model in 4-bit NF4 and trains 16-bit LoRA adapters on top. During each forward/backward pass, the touched 4-bit weights are dequantized to 16-bit on the fly, used, and discarded — gradients flow through the frozen quantized weights but are only applied to the adapters.

Sixteen levels is all 4 bits buy you. Spend them evenly across the range (like INT4 does) and you waste levels where weights are rare. Neural weights are approximately normally distributed — bunched near zero, thin tails — so QLoRA's NormalFloat-4 places its 16 levels at the quantiles of a normal distribution: each level owns an equal share of the actual weights.

It's the information-theoretic answer to "where do I put 16 points to describe a bell curve?" — dense levels where weights crowd, sparse levels in the tails. In ablations, NF4 beat standard FP4 consistently — the difference between 4-bit-that-degrades and 4-bit-that-doesn't.

Block-wise quantization stores a scale constant per block of weights — and at QLoRA's small block size (64, chosen for accuracy), those 32-bit constants add up: ~0.5 bits per parameter of pure overhead, gigabytes at 65B scale. QLoRA's cheeky fix: quantize the quantization constants too — the per-block scales get their own 8-bit quantization with a second-level constant.

It saves ~0.37 bits/param — about 3 GB on a 65B model. Not glamorous, but it's the difference between fitting on the card and not. Systems papers are won in the decimals.

Even with everything above, training memory spikes — a long sequence's activation checkpointing can momentarily demand gigabytes, and one spike OOMs an otherwise-fitting run. QLoRA uses paged optimizers: optimizer states live in unified memory that the NVIDIA driver pages between GPU and CPU on demand, like OS virtual memory. Spikes swap out; training hiccups instead of dying.

Together the three tricks turn "fits on average" into "fits, period" — the practical difference between a paper result and a tool people actually run overnight on one machine.

The ablation scale is its own argument: because each run was suddenly cheap, the authors could fine-tune over a thousand models to map what matters (data quality trumps data quantity, was one finding). Cheap iteration isn't just economical — it changes what science gets done.

QLoRA is why "fine-tune it on your own data" became a weekend project instead of an infrastructure program. Shipped into bitsandbytes and Hugging Face PEFT within weeks, it became — and remains — the default recipe behind most open-model fine-tunes you've ever downloaded. The NF4-frozen-base pattern also normalized a bigger idea: quantization isn't just a deployment trick, it's a training-time tool.

Conceptually it completed LoRA's argument. LoRA said: the update is low-rank, so shrink the training. QLoRA added: the base is redundant, so shrink the storage. Between them, the two papers cut fine-tuning's cost by orders of magnitude in each dimension — and the open-source model ecosystem is the compound interest.