Vibe Engines
YouTube

Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

Transformer Math

The napkin formulas behind every scaling, memory and cost conversation — params, FLOPs, KV cache, VRAM.

A transformer’s cost follows from a few shape numbers: parameters N, layers L, hidden size d, context length and bytes per value. From those, napkin formulas give compute, memory and speed to within a factor good enough for planning. Two facts organize everything below. Compute scales with parameters × tokens: about 6N FLOPs per token to train and 2N to generate. And generating one token at a time is limited by memory bandwidth, not compute, because every new token rereads all the weights.

Figures are approximate on purpose. Use them to compare options, then measure.

embedvocab·d× L layersattention4·d²Q K V OMLP8·d²up, down12·L·d² paramsheadd·vocabKV cache: 2·n_kv·d_head·bytesper layer, per token in contexttrain ≈ 6N FLOPs/tokgenerate ≈ 2N/tok+ read all N weights
Parameters live in the repeated layers and the embeddings; compute follows parameters × tokens; the KV cache grows with every token of context and every concurrent user.

Parameters & compute

Params (dense)
≈ 12 · L · d² (L layers, d hidden) — attention 4d², MLP 8d²
Training FLOPs
≈ 6 · params · tokens (2 forward + 4 backward)
Inference FLOPs
≈ 2 · params per generated token (dense)
Chinchilla-optimal
≈ 20 tokens per parameter (modern models train far past it)
Compute-optimal scaling
params and tokens both grow with ≈ √compute: 4× compute → 2× model, 2× data
MoE active params
compute follows active params; memory follows total
Embedding params
vocab × d, doubled when input and output embeddings are untied — about 1B in Llama 3 8B
Attention FLOPs
add ≈ 2 · L · n_ctx · d per token (Kaplan et al.) — small next to 2N until context is long

Architecture shapes

d_head = d / n_heads
usually 128; attention scores are scaled by 1/√d_head
FFN width
classic d_ff ≈ 4d; SwiGLU uses 3 matrices with d_ff ≈ 8d/3 to keep ≈ 8d² params
MHA · GQA · MQA
K/V heads = query heads · a few groups (often 8) · one — fewer K/V heads, smaller cache
Llama 3 8B
32 layers, d = 4096, 32 query and 8 KV heads, d_ff = 14336, 128K vocabulary
Llama 3 70B
80 layers, d = 8192, 64 query and 8 KV heads, d_ff = 28672

Memory

Weights
params × bytes — FP16 = 2, INT8 = 1, INT4 = 0.5 B/param
Rule of thumb
7B FP16 ≈ 14 GB · 70B FP16 ≈ 140 GB · INT4 quarters it
KV cache / token
2 · L · (d · kv_heads / heads) · bytes — GQA divides it
KV cache, worked
Llama 3 8B at FP16: 2·32·8·128·2 B = 128 KiB per token, 1 GiB at 8K context
KV cache × batch
32 users at 8K on Llama 3 8B ≈ 32 GiB — twice the weights
Full fine-tune (AdamW)
≈ 16 B/param: 16-bit weights and grads, plus FP32 master copy and two moments
LoRA / QLoRA
≈ 2.5 / 0.75 B/param — the base dominates
Activations
grow with batch × sequence; checkpointing trades ~33% more compute for far less memory
ZeRO-3 / FSDP
shard weights, gradients and optimizer state over N GPUs: ≈ 16 B/param ÷ N each

Throughput & latency

Decode speed ceiling
≈ memory bandwidth ÷ model bytes (bandwidth-bound)
Prefill
compute-bound, parallel over prompt — thousands of tok/s
TTFT
≈ queue + prompt_tokens ÷ prefill_rate (+ network)
TPOT
time per output token ≈ model bytes ÷ bandwidth; total latency = TTFT + TPOT × tokens
Speculative decoding
draft k tokens, verify in 1 pass — 2-3× decode, identical output
Batching
raises throughput, not per-user speed — decode stays bandwidth-bound
Compute-bound batch
decode becomes compute-bound near batch ≈ FLOPs ÷ bandwidth, roughly 300 on an H100
Tensor parallelism
split every layer across GPUs in one node; needs NVLink-class bandwidth
Pipeline parallelism
split layers into stages across GPUs or nodes; costs idle “bubble” time

Tokens & text

1 token ≈ 4 characters
about ¾ of an English word; code and non-English text take more tokens
1,000 tokens
≈ 750 English words, roughly a page and a half of prose
128K context
≈ 96,000 words — a long novel in one prompt
Perplexity
exp(loss) with loss in nats — loss 2.0 ≈ perplexity 7.4
Temperature
logits ÷ T before softmax: T below 1 sharpens, above 1 flattens; T → 0 is greedy

Training at scale

GPU-hours
6·N·D ÷ (peak FLOPs × MFU) ÷ 3600; MFU of 30–50% is good
With recomputation
≈ 8 · N · D — the extra forward pass pays for checkpointed activations
Llama 3 8B data
15T+ tokens ≈ 1,900 tokens per parameter, about 90× the Chinchilla ratio
Llama 3.1 405B
≈ 3.8 × 10²⁵ training FLOPs; Meta reports about 30.8M H100 GPU-hours

Cost

API cost
in_tok/1M × in_price + out_tok/1M × out_price; output usually 4–8× the input price
Blended $/1M
(ratio · in + out) / (ratio + 1) at in:out ratio
History growth
chat resends history each turn — input tokens grow ~quadratically
Prompt caching
cached input bills at a steep discount (0.1× on Anthropic’s API); keep stable parts first
Self-host $/1M tokens
$/hr ÷ (tok/s × 3600) × 1e6 — $2/hr at 1,000 tok/s ≈ $0.56
Batch API
non-urgent requests at half price on the major APIs, results within 24 hours
Reasoning tokens
thinking is billed as output — a short answer can carry thousands of hidden tokens

Worked example: Llama 3 8B from its config file

Start from the published shape and derive everything else. The shortcut and the real count disagree, and the reason is instructive.

Shape: L = 32, d = 4096, 32 query heads, 8 KV heads, d_head = 128,
       d_ff = 14336 (SwiGLU), vocab = 128,256, untied embeddings

Attention per layer:  Q, O: 2 × 4096 × 4096        =  33.6M
                      K, V: 2 × 4096 × (8 × 128)   =   8.4M   (GQA)
MLP per layer:        3 × 4096 × 14336             = 176.2M
Per layer:                                         ≈ 218.1M
32 layers:                                         ≈ 6.98B
Embeddings:           2 × 128,256 × 4096           ≈ 1.05B
Total:                                             ≈ 8.03B  (the published count)
12·L·d² shortcut:     12 × 32 × 4096²              ≈ 6.44B

The shortcut is 20% low. GQA shrinks attention to 2.5d² instead of 4d², but the wide SwiGLU MLP (d_ff = 3.5d) holds 10.5d² instead of 8d², and the formula leaves out the 1.05B embedding parameters entirely. With the true count:

Weights, BF16:       8.03e9 × 2 B                  = 16.1 GB
KV cache, BF16:      2 × 32 × 8 × 128 × 2 B        = 128 KiB/token → 1 GiB at 8K
Generate:            2 × 8.03e9                    ≈ 16 GFLOPs per token
  attention at 8K:   2 × 32 × 8192 × 4096          ≈ 2.1 GFLOPs more (+13%)
Train on 15T tokens: 6 × 8.03e9 × 15e12            ≈ 7.2e23 FLOPs
  ÷ (989e12 × 0.40 MFU) ÷ 3,600                    ≈ 507K H100-hours

Meta’s model card reports about 1.3M H100-hours for Llama 3 8B, roughly 2.6× this estimate. Treat 6·N·D at a hopeful MFU as a floor, not a forecast.

Recipes

Will a 70B model fit, and how fast will it generate?

Weights (FP16):   70e9 × 2 B             = 140 GB  → two 80 GB GPUs
Weights (INT4):   70e9 × 0.5 B           = 35 GB   → one 48–80 GB GPU
KV cache (FP16):  2 × 80 × 1,024 × 2 B   = 320 KiB per token
                  × 8,192 tokens         ≈ 2.5 GiB per sequence
Decode ceiling:   3.35 TB/s ÷ 35 GB      ≈ 95 tokens/s (INT4, one H100)

Bandwidth divided by model bytes is a ceiling: real systems land below it, and reading the KV cache lowers it further as the context grows.

Training compute and cost for a 7B model

Tokens (Chinchilla):  20 × 7e9              = 140e9 tokens
FLOPs:                6 × 7e9 × 140e9        = 5.9e21
Effective H100:       989e12 × 0.40 MFU      = 4.0e14 FLOP/s
GPU-seconds:          5.9e21 ÷ 4.0e14        = 1.5e7 s ≈ 4,100 GPU-hours
Cost at ~$2.50/hr:                           ≈ $10,000

Production models train far past 20 tokens per parameter (Llama 3 8B saw about 1,900), which multiplies this figure by roughly 100.

API cost of one chat conversation

Conversation:    8 turns, each adding 1,500 input and 400 output tokens
Input per turn:  history is resent: 1.5K, 3.4K, 5.3K … 14.8K ≈ 65K in total
Output:          8 × 400 = 3,200 tokens
Example price:   $3 per 1M input, $15 per 1M output
Cost:            65K × $3/1M + 3.2K × $15/1M ≈ $0.20 + $0.05 ≈ $0.24
With caching:    reads at 0.1×, writes at 1.25× the input price ≈ $0.12

Resending history makes input grow with the square of the turn count, which is why long conversations dominate the bill and why caching the shared prefix halves it here.

Self-hosting versus an API

GPU:               1× H100 at ~$2.50/hr ≈ $1,825 per month
Throughput:        assume 2,000 output tokens/s across a busy batch (measure yours)
Monthly capacity:  2,000 × 3,600 × 730 ≈ 5.3B tokens
Cost per 1M:       $1,825 ÷ 5,256 ≈ $0.35, only while the GPU stays busy
At 10% load:       ≈ $3.50 per 1M

Utilization decides the answer: a GPU that sits idle 90% of the time costs ten times more per token, which is usually when a per-token API wins.

When to use it, and when not

Use these formulas before you spend money: picking a GPU, checking that a model fits, estimating a training budget, testing a vendor’s tokens-per-second claim, or comparing an API bill with self-hosting. They give the right order of magnitude in a minute, which is usually enough to rule options out.

Measure instead when:

  • two options land within about 30% of each other, because the approximations are coarser than that;
  • you serve with continuous batching and a paged KV cache, where throughput depends on the mix of prompt and output lengths;
  • contexts are long enough that attention FLOPs and KV-cache reads dominate, which the 2N rule leaves out;
  • the model is a mixture of experts, where routing and expert placement decide real speed;
  • you need p99 latency, which no average-based formula predicts.

Trade-offs and limits

ShortcutWhat it ignoresOff most when
12·L·d²embeddings, GQA, SwiGLU widthsmall models: Llama 3 8B gives 6.4B, not 8.03B
6·N·Dattention FLOPs, recomputationlong sequences; checkpointing makes it ≈ 8·N·D
2·N per tokenattention over the contextlong contexts: +13% at 8K for Llama 3 8B
bandwidth ÷ bytesKV-cache reads, kernel overheadlarge batches with long contexts
20 tokens per parameterserving costany model that will serve heavy traffic

Each shortcut drops a term to keep the arithmetic in your head. When a decision hinges on the dropped term, write it back in before trusting the answer.

Common mistakes

  • Mixing up total and active MoE parameters. Mixtral 8x7B has 46.7B parameters but uses 12.9B per token. It needs memory for all 46.7B (about 93 GB at FP16) and computes like a 13B model.
  • Using the MHA KV formula on a GQA model. For Llama 3 70B, counting all 64 heads gives 2.5 MiB per token; with its 8 KV heads the real figure is 320 KiB, 8× less.
  • Mixing GiB and GB. 128 KiB × 8,192 tokens is 1 GiB, which is 1.07 GB. Across 48 users that is 51.5 GB, not 48: enough to run a tight card out of memory.
  • Timing a prompt with the decode formula. Prefill is compute-bound. For an 8B model on an H100, 2,000 prompt tokens at the 209 tok/s decode ceiling would take 9.6 s; prefill actually needs about 2·N·2,000 ÷ (FLOPs × MFU) ≈ 0.08 s.
  • Sizing a fine-tune by the weights. A 7B model is 14 GB at FP16, but a full AdamW fine-tune needs about 112 GB once gradients, the FP32 master copy and two optimizer moments are added.

Check yourself

Roughly how many parameters does a dense model with 24 layers and d = 2048 have, before embeddings?

12 × 24 × 2048² ≈ 1.2B. Add vocab × d (twice if untied) for the embeddings.

How big is Llama 3 70B’s KV cache per token with an FP8 cache, and for one 32K-token sequence?

2 × 80 layers × 8 KV heads × 128 × 1 B = 160 KiB per token, so 160 KiB × 32,768 = 5 GiB per sequence.

A 1B-parameter model is trained Chinchilla-optimally. How many tokens and how many FLOPs?

20 × 1e9 = 20B tokens, and 6 × 1e9 × 20e9 = 1.2 × 10²⁰ FLOPs.

A server prefills at 5,000 tokens/s and generates at 25 ms per output token. What are TTFT and total latency for a 10,000-token prompt and a 300-token answer, ignoring queueing?

TTFT = 10,000 ÷ 5,000 = 2 s. Total = 2 + 300 × 0.025 = 9.5 s. The answer length, not the prompt, dominates here.

Questions about Transformer Math Cheat Sheet

How much GPU memory do I need to run an LLM?

Start with the weights: parameters times bytes per parameter, so 2 bytes at FP16 or BF16, 1 byte at INT8 and about half a byte at 4-bit. A 7B model needs roughly 14 GB at FP16 and 4 to 5 GB at 4-bit, while a 70B model needs about 140 GB at FP16 and 35 to 40 GB at 4-bit. Then add the KV cache, which grows with context length and with the number of concurrent requests and can exceed the weights at long contexts, plus 10 to 20 percent for activations and framework overhead.

Why is LLM inference memory-bandwidth bound?

Generating each new token requires reading every weight of the model from GPU memory, but the arithmetic per weight is tiny: about two floating-point operations per parameter for one sequence. A modern GPU can do hundreds of operations in the time it takes to fetch one byte, so during decoding the compute units mostly wait for memory. That is why tokens per second track memory bandwidth divided by model size, why quantization speeds up decoding, and why batching raises throughput: each weight fetched from memory then serves many sequences.

What does Chinchilla optimal mean?

It refers to DeepMind’s 2022 Chinchilla study, which found that for a fixed training compute budget, loss is minimized by scaling model size and training tokens roughly equally, at about 20 tokens per parameter. Many earlier models had been too large for the data they saw. Today’s models deliberately train far beyond that point, often a hundred times more tokens per parameter, because a smaller model trained longer costs more to train but much less to serve, and serving costs dominate once a model has millions of users.

How do you calculate the KV cache size?

For each token, every layer stores one key vector and one value vector per key-value head. Multiply 2 for keys and values by the number of layers, the number of key-value heads, the head dimension and the bytes per value. Llama 3 8B has 32 layers and 8 key-value heads of dimension 128, so at FP16 that is 2 times 32 times 8 times 128 times 2 bytes, or 128 KiB per token. That comes to about 1 GiB for an 8K context, multiplied again by the number of concurrent sequences.