LLM Sampling Simulator
An interactive LLM sampling playground. Adjust temperature, top-p (nucleus) and top-k and watch the next-token probability distribution reshape in real time — from sharp and near-greedy at low temperature to flat and creative at high temperature, with the long tail trimmed by top-p and top-k. The exact softmax math that turns raw logits into the choices a language model actually samples from.
The cat sat on the___
Sampling from 6 of 6 tokens · greedy pick would be mat. Balanced — the model’s natural distribution.
A language model does not output words — it outputs a score (a logit) for every token in its vocabulary, and sampling is how those scores become one actual choice. The logits are first turned into probabilities with the softmax function, and temperature controls how sharp that distribution is by dividing every logit by the temperature before the softmax. A temperature below 1 exaggerates the differences, so the top token dominates and output is more deterministic; a temperature above 1 shrinks the differences toward a flat, uniform distribution, so output is more random and surprising. At temperature near 0 the model is effectively greedy — it always takes the single highest-scoring token. This playground runs that exact softmax over a small set of candidate next tokens so you can watch the bars sharpen and flatten as you move the slider.
Temperature reshapes the whole distribution but never removes the long tail of unlikely tokens, and sampling from that tail is what produces incoherent text. That is what truncation fixes. Top-k keeps only the k highest-probability tokens and discards the rest. Top-p, or nucleus sampling, is adaptive: it sorts tokens by probability and keeps the smallest group whose cumulative probability reaches p, so it keeps few candidates when the model is confident and many when it is unsure. After either filter the surviving probabilities are renormalized to sum to 1 and the model samples from just those. The two are commonly combined — top-k sets a hard ceiling on the number of candidates and top-p trims within it — which is why most APIs expose all three knobs at once.
Read this widget as a scaled-down version of a single decoding step. Watch how lowering temperature and tightening top-p both concentrate the mass on the leading token, but in different ways: temperature squeezes the whole shape continuously, while top-p and top-k chop the tail off at a threshold. A practical starting point is a low temperature for factual work and a temperature around 0.7 to 1.0 with top-p near 0.9 for creative work. Two honest caveats: a real model chooses among tens of thousands of tokens at every step, not the handful shown here, so this is intuition rather than a literal reproduction; and the candidate scores here are fixed illustrative values, whereas a real model recomputes them from scratch for every position given everything written so far. The math shown is exact for these inputs; the right settings still depend on your model and task.
How it works
- Softmax with temperature: p_i = exp(logit_i / T) divided by the sum of exp(logit_j / T).
- Top-k keeps the k highest-probability tokens; top-p keeps the smallest set reaching cumulative p.
- Filtered-out tokens are dropped and the survivors renormalized to sum to 1.
- Move the sliders to watch the next-token distribution reshape in real time.
Frequently asked questions
What does temperature do in LLM sampling?
Temperature scales the logits before the softmax by dividing them by the temperature value. A low temperature below 1 sharpens the distribution toward the single most likely token, making output more deterministic; a high temperature above 1 flattens the distribution toward uniform, making output more random and varied. Temperature approaching 0 is effectively greedy decoding — it always picks the top token.
What is top-p (nucleus) sampling?
Top-p, or nucleus sampling, keeps the smallest set of most-likely tokens whose cumulative probability reaches p, then samples from just that set after renormalizing. Top-p of 0.9 means “consider the tokens that together cover 90% of the probability mass and ignore the long tail.” It adapts the cutoff to the shape of each distribution instead of using a fixed count.
Top-p vs top-k — what is the difference?
Top-k always keeps a fixed number k of the highest-probability tokens; top-p keeps a variable number that depends on how peaked the distribution is. On a confident step top-p might keep only one token, while on an uncertain step it keeps many. They are often combined: top-k caps the maximum number of candidates and top-p trims within that cap.
What sampling settings should I use?
For factual or deterministic tasks, use a low temperature around 0 to 0.3, or greedy decoding. For creative writing, use a higher temperature around 0.7 to 1.0 with top-p near 0.9. Extreme temperature with no top-p or top-k lets rare, incoherent tokens through; top-p and top-k keep that tail in check while temperature controls the spread among the plausible tokens.