Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path
Fine-Tuning Cost Estimator
Pick a method (QLoRA / LoRA / full AdamW), model size, dataset and GPU, and get the napkin answer: training memory, how many cards you need to fit, GPU-hours from the 6·P·tokens FLOPs rule, wall-clock time and the rental bill — with the QLoRA-vs-full memory gap made painfully visible.
A fine-tuning budget has two separate questions in it: what hardware can hold the job, and how long that hardware runs. This estimator answers both from five inputs, the method, model size, dataset size, epochs and GPU, and shows that the method mostly changes the first answer while the dataset drives the second.
What it computes
memory = P × bytes per parameter / 1,024³ GB
GPUs = ⌈ memory / (VRAM × 0.85) ⌉
tokens = dataset × epochs
GPU-hours = 6 × P × tokens / (TFLOPs × 10¹² × 0.35) / 3,600
wall-clock = GPU-hours / GPUs
cost = GPU-hours × price per GPU-hour- P is the parameter count. Bytes per parameter is 16 for full fine-tuning with AdamW, 2.5 for LoRA on a 16-bit base and 0.75 for QLoRA on a 4-bit base.
- 6 × P × tokens is the standard estimate of training FLOPs: about 2 per parameter per token forward and 4 backward.
- 0.35 is the assumed utilization: real fine-tuning jobs reach a fraction of a GPU’s peak.
- The GPU table uses dense BF16 throughput, 165, 121, 312 and 989 TFLOPs for the RTX 4090, L4, A100 and H100, with illustrative rental prices of $0.50, $0.70, $2.50 and $3.50 per hour.
A worked example
The defaults are QLoRA on an 8B model, 50 million tokens, 3 epochs, on an A100 80 GB.
- Memory:
8 × 10⁹ × 0.75 = 6.0 billion bytes = 5.6 GB. One card offers80 × 0.85 = 68 GB, so 1×, fits on a single card. - Tokens:
50M × 3 = 150Mtrained. - FLOPs:
6 × 8 × 10⁹ × 1.5 × 10⁸ = 7.2 × 10¹⁸. - Throughput:
312 × 10¹² × 0.35 = 1.09 × 10¹⁴FLOPs per second, so7.2 × 10¹⁸ / 1.09 × 10¹⁴ = 65,934 s = 18.3 GPU-hours. - Cost:
18.3 × $2.50 = $45.79, shown as $46, and the ideal wall-clock is 18.3 h.
| Same job, 8B | Memory | GPUs | Wall-clock | Cost |
|---|---|---|---|---|
| QLoRA, A100 | 5.6 GB | 1 | 18.3 h | $46 |
| LoRA, A100 | 18.6 GB | 1 | 18.3 h | $46 |
| Full, A100 | 119.2 GB | 2 | 9.2 h | $46 |
| QLoRA, H100 | 5.6 GB | 1 | 5.8 h | $20 |
| QLoRA, RTX 4090 | 5.6 GB | 1 | 34.6 h | $17 |
On one GPU type the bill does not change with the method; only the card count and the clock do. At 70B the gap becomes decisive: a full fine-tune needs 1,043 GB and 16 A100s, while QLoRA needs 48.9 GB and one.
Assumptions and limits
- Activations are not counted. Long sequences and large batches add memory beyond the bytes-per-parameter figures, sometimes a lot, so a job shown as fitting can still run out of memory.
- Perfect scaling. Multi-GPU runs lose time to communication, often 10 to 30%, and the tool ignores it.
- One FLOP formula for every method. Adapter methods can save some backward-pass compute and QLoRA adds dequantization work; neither is modeled.
- Fixed utilization and prices. 35% and the hourly rates are planning defaults. Real utilization and market prices vary widely by provider.
- One run. Experiments, failed runs and evaluation are not included.
How to use the result
- Pick the method by memory: use the cheapest card class the job fits on, then compare cost and wall-clock across GPUs.
- Faster cards can be cheaper. A higher hourly rate can still win if throughput rises faster, as the H100 row shows.
- Budget three to five times the single-run cost for hyperparameter trials and reruns.
- Spend on data before epochs. Cost is linear in tokens trained; doubling epochs doubles the bill and rarely fixes weak data.
How it works
- GPU-hours = 6 · params · tokens ÷ (TFLOPs × 35% utilization).
- Memory: full ≈ 16 B/param, LoRA ≈ 2.5, QLoRA ≈ 0.75 (activations excluded).
- GPUs-to-fit uses 85% usable VRAM; parallelism overhead not included.
- GPU prices are illustrative rental rates; substitute your own.
Frequently asked questions
How are GPU-hours estimated?
From the standard training-compute rule: total FLOPs are about 6 times the parameter count times the tokens trained, covering the forward and backward passes. The tool divides that by the GPU’s dense BF16 throughput at 35% realized utilization and converts seconds to hours. Tokens trained are the dataset size times the number of epochs. It is napkin math, since sequence length, batch size, parallelism and data loading all move real runs, so treat it as roughly plus or minus 40%.
How much does it cost to fine-tune an 8B model?
With the tool’s defaults, QLoRA on 50 million tokens for 3 epochs, the run is 150 million trained tokens and about 18.3 GPU-hours on an A100, or about $46 at the illustrative $2.50 per hour rate. The same run is about $20 on an H100 at $3.50 per hour, because the H100 is roughly three times faster, and about $17 on an RTX 4090 at $0.50 per hour, though it takes 34.6 hours there. Substitute the rates you can actually rent at.
Why does QLoRA need so much less memory than full fine-tuning?
Full fine-tuning with AdamW holds the weights, the gradients and two optimizer moments for every parameter, roughly 16 bytes per parameter in mixed precision. QLoRA freezes the base model in 4-bit, about half a byte per parameter, and trains only small 16-bit adapters, which this tool models as 0.75 bytes per parameter overall. That is over 20 times less memory: a 70B model needs about 1,043 GB for a full fine-tune but about 49 GB for QLoRA, before activations.
Does LoRA or QLoRA train faster than full fine-tuning?
Not by much per token. Adapter methods still run the full forward pass and backpropagate through every layer of the frozen base, so they need most of the compute of full fine-tuning. They skip the gradients for the frozen weights, which can save up to about a third, but QLoRA spends some of that back dequantizing 4-bit weights. This tool charges all three methods the same 6 times parameters times tokens, so the method changes the GPUs you need, not the bill.
How many epochs should I plan for?
Instruction fine-tunes typically converge in one to three epochs, and more passes usually overfit a small dataset, memorizing examples instead of learning the pattern. For budgeting, three epochs is a sensible default, and the tool scales cost linearly with it. If quality is not there after that, the fix is almost always better or more varied data, not more passes over the same examples, so hold back an evaluation set to check.