LLM Sampling & Decoding
Every knob that shapes how a language model picks its next token — and sensible presets.
Parameters
- temperature
- Flatten/sharpen the distribution. 0 = greedy, 1 = as-is, >1 = wilder.
- top-k
- Keep only the k highest-probability tokens, then renormalize.
- top-p (nucleus)
- Keep the smallest set whose probability sums to p (e.g. 0.9).
- min-p
- Drop tokens below a fraction of the top token’s probability.
- frequency penalty
- Lower the odds of tokens by how often they’ve appeared.
- presence penalty
- Lower the odds of any token that has appeared at all.
- max tokens
- Hard cap on the length of the generated output.
- stop sequences
- Strings that end generation when produced.
Presets
- Deterministic
temperature 0— tools, extraction, tests- Balanced
temp 0.7, top-p 0.9— general chat- Creative
temp 1.0+, top-p 0.95— brainstorming, prose