Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path
LLM Sampling Simulator
An interactive LLM sampling playground. Adjust temperature, top-p (nucleus) and top-k and watch the next-token probability distribution reshape in real time — from sharp and near-greedy at low temperature to flat and creative at high temperature, with the long tail trimmed by top-p and top-k. The exact softmax math that turns raw logits into the choices a language model actually samples from.
The cat sat on the___
Sampling from 6 of 6 tokens · greedy pick would be mat. Balanced — the model’s natural distribution.
A language model never outputs a word directly: it scores every token in its vocabulary, and sampling turns those scores into one choice. This simulator runs a single decoding step for the prompt "The cat sat on the ___" over six candidate tokens with fixed, illustrative logits, so you can watch temperature, top-p and top-k reshape the distribution.
What it computes
The logits are mat 3.2, floor 2.1, couch 1.8, roof 1.5, table 1.2 and chair 0.9. Each slider change reruns four steps:
p_i = exp((logit_i − max) / T) ÷ Σ_j exp((logit_j − max) / T)
ranked = tokens sorted by p_i, highest first
kept = walk down ranked; stop after k tokens, or after the token
that brings the running total of p_i to at least top_p
final_i = p_i ÷ Σ(p_j over kept tokens), and 0 for dropped tokens- T is the temperature, floored at 0.01. Subtracting the largest logit first changes nothing mathematically; it only keeps the exponentials from overflowing.
- top_k caps how many tokens survive, and top_p closes the nucleus once the kept probability reaches p.
- The greedy pick is the highest-probability token, which is mat at every temperature, because temperature rescales scores without reordering them.
Kept tokens show a percentage, rounded to a whole number at 10% or more and to one decimal below that. Dropped tokens show a dash. The mood line reads near-greedy at T ≤ 0.2, focused below 0.8, balanced up to 1.2 and adventurous above.
A worked example
The defaults are temperature 1.00, top-p 1.00 and top-k 6, so no filter applies.
- Subtract the maximum, 3.2:
0, −1.1, −1.4, −1.7, −2.0, −2.3. - Exponentiate:
1, 0.333, 0.247, 0.183, 0.135, 0.100, which sum to1.998. - Divide by the sum: mat 50%, floor 17%, couch 12%, roof 9.1%, table 6.8%, chair 5.0%.
- The readout says sampling from 6 of 6 tokens, greedy pick mat, balanced.
| Setting (others at default) | Tokens kept | Result |
|---|---|---|
| Temperature 0.5 | 6 | mat 81%, floor 9.0%; focused |
| Temperature 0.2 | 6 | mat 99%; near-greedy |
| Temperature 2.0 | 6 | mat 31%, chair 9.9%; adventurous |
| Top-p 0.9 | 5 | chair dropped; mat 53% |
| Top-p 0.5 | 1 | mat 100% |
| Top-k 3 | 3 | mat 63%, floor 21%, couch 16% |
For top-p 0.9 the running total goes 0.501, 0.667, 0.791, 0.882 and then 0.950 after table, which crosses 0.9, so chair is cut and the five survivors are divided by 0.950.
Assumptions and limits
- Six tokens, one step. A real model scores a vocabulary of tens of thousands of tokens and recomputes the logits for every position, so this is intuition, not a reproduction.
- Filter order. The nucleus here is measured on the temperature-scaled probabilities of all six tokens. Some libraries apply top-p after renormalizing the top-k survivors, which can keep fewer tokens: temperature 1.0, top-k 3 and top-p 0.8 keeps three here but would keep two that way.
- No random draw. The widget shows the distribution a sampler would draw from; it does not roll the dice.
- Other controls are absent. Min-p, repetition and frequency penalties and logit bias also change the distribution in real APIs.
How to use the result
- Match the setting to the task. Tasks with one right answer want low temperature or greedy decoding; open-ended writing tolerates more spread, usually with top-p trimming the tail.
- Change one knob at a time. Lowering temperature and tightening top-p both concentrate probability on the leader, so moving both hides which one helped.
- Watch the kept count. If top-p keeps a single token, sampling is effectively greedy on that step whatever the temperature.
- Never sample the raw tail. A high temperature with no top-p or top-k gives every unlikely token a real chance, which is where incoherent text comes from.
How it works
- Softmax with temperature: p_i = exp(logit_i / T) ÷ Σ exp(logit_j / T).
- Top-k keeps the k most likely tokens; top-p keeps the smallest set whose probability reaches p.
- Filtered tokens are dropped and the survivors renormalized to sum to 1.
- One fixed next-token step over six illustrative logits, recomputed as you move the sliders.
Frequently asked questions
What does temperature do in LLM sampling?
Temperature divides every logit before the softmax. Below 1 it exaggerates the gaps between scores, so the top token takes more of the probability; above 1 it shrinks them, so the distribution flattens and rarer tokens get picked more often. In this simulator the top token, mat, has 50% at temperature 1.0, 81% at 0.5 and 99% at 0.2, but only 31% at 2.0. Temperature never changes the ranking, only how much the leader dominates.
What is top-p (nucleus) sampling?
Top-p keeps the smallest group of most-likely tokens whose probabilities add up to at least p, discards the rest, and rescales the survivors to sum to 1. Because the group size follows the shape of the distribution, it keeps few tokens when the model is confident and many when it is unsure. At temperature 1.0 here, top-p 0.9 keeps five of the six tokens and drops chair, while top-p 0.5 keeps only mat.
What is the difference between top-p and top-k?
Top-k always keeps a fixed number of the highest-probability tokens, however the probability is spread. Top-p keeps a variable number that depends on how peaked the distribution is, so on a confident step it might keep one token and on an uncertain step many. In this simulator, top-k 3 keeps mat, floor and couch at 63%, 21% and 16%. When both are set, the tool stops at whichever limit is reached first.
Is temperature 0 the same as greedy decoding?
Mathematically yes: as temperature approaches 0 the softmax puts all the probability on the highest-scoring token, which is exactly the greedy choice. The simulator floors temperature at 0.01 in its math and the slider starts at 0.05, where mat shows 100%. In practice many APIs treat 0 as greedy, but repeated calls can still differ slightly, because the logits themselves can vary with floating-point effects in batched inference.
What temperature and top-p should I use?
Common starting points are a low temperature, or greedy decoding, for extraction, classification, code and other tasks with one right answer, and a moderate temperature with top-p around 0.9 for brainstorming and creative writing. Avoid a high temperature with no truncation, which lets the unlikely tail through. Change one knob at a time, since temperature and top-p both concentrate probability, and check your provider defaults, which differ.
Why are probabilities renormalized after top-k or top-p?
A sampler has to draw from a distribution that sums to 1, so once some tokens are removed the survivors are divided by their combined probability. That rescaling keeps their relative odds unchanged. With top-p 0.9 at temperature 1.0, the five kept tokens hold 95% of the original probability, so each is divided by 0.95 and mat rises from 50% to 53%. The removed tokens show a dash rather than 0%.