Prompt Caching
Reusing the processed prefix of a prompt so you don’t re-pay to encode it every turn.
Prompt caching lets a provider skip re-processing a stable prompt prefix (system message, tools, long context) across calls, cutting latency and cost. It rewards ordering prompts so the unchanging parts come first.
Worked example: caching the KV state of a stable prompt prefix (system prompt + tools + shared history) lets the provider skip re-encoding it next call — cutting latency and cost for the repeated head. Gotcha: the cache only hits on an exact prefix match, so keep the stable part first and byte-identical; change one token early and the whole cache misses, which is why prompt structure (stable → variable) matters.