The Dice Loader.

The three dials — what each does

1. Softmax first. Logits become probabilities: p_i = e^(z_i / T) / Σ e^(z_j / T). That divisor T is temperature.
2. Temperature = sharpness. T < 1 concentrates mass on the top tokens (safe, repetitive); T > 1 spreads it out (diverse, risky); T → 0 is greedy.
3. Top-k = a hard cap. Keep the k most likely tokens, zero the rest, renormalize. Fixed count, ignores the distribution's shape.
4. Top-p = an adaptive cap. Keep the smallest set whose probabilities sum to p, renormalize. Few survivors when confident, many when unsure.

The whole thing in code

import numpy as np

def sample(logits, temp=1.0, k=None, p=None):
    z = logits / temp                       # temperature
    probs = np.exp(z - z.max())
    probs /= probs.sum()                    # softmax

    order = probs.argsort()[::-1]           # high → low
    keep = set()
    cum = 0.0
    for rank, i in enumerate(order):
        if k is not None and rank >= k:     # top-k cap
            break
        keep.add(i); cum += probs[i]
        if p is not None and cum >= p:      # top-p cap
            break

    masked = np.array([probs[i] if i in keep else 0
                       for i in range(len(probs))])
    masked /= masked.sum()                  # renormalize
    return np.random.choice(len(masked), p=masked)