Inference
Running a trained model forward to produce an output — as opposed to training, which updates its weights.
Inference is a model doing its job: given fixed, already-trained weights, compute a forward pass to produce a prediction or generate tokens. It’s a fundamentally different engineering problem from training — optimized for low latency and high request throughput rather than gradient computation — which is why techniques like quantization, continuous batching, and speculative decoding exist specifically for the inference side of the stack.