The bill is
just tokens.

Every LLM API bills the same way: per token, with prices quoted per million tokens. A request has two token counts — input (everything you send: system prompt, context, the user's message) and output (what the model generates) — and each has its own price. The cost of one request is just input tokens times input price plus output tokens times output price, divided by a million. That's it. No hidden magic; the whole bill is that sum, repeated across every call.

The reason bills surprise people is that they never estimate that sum at scale. A single request costing $0.01 feels free; a million requests a day is $10,000. Cost engineering is nothing more than doing this arithmetic before the invoice, instrumenting the token counts you actually send, and then pulling the levers the equation reveals. And the equation reveals a lopsided one: the two prices are not equal.

Output tokens are typically priced several times higher than input tokens — a 5× ratio is common. That asymmetry changes everything about where cost hides. Two requests with the same total token count can differ 2× in price depending on how the tokens split between input and output. A verbose model that restates the question, thinks out loud, and pads its answer is quietly the most expensive thing in your system.

So the highest-leverage cost cut is almost always the same: make outputs shorter. Ask for concise answers, set a sensible max_tokens, use structured outputs so the model emits only the fields you need, and don't make it echo the input back. This single lever routinely beats the fancy ones, because it attacks the most expensive term in the equation directly. Input is cheaper, so a big fixed context hurts less than a big generation — and the next section shows how to make that fixed context nearly free.

Many applications send the same long prefix on every call — a big system prompt, a style guide, a document you keep asking about. Prompt caching stores that prefix so later requests pay only a small fraction of the input price for it (a cache read is often ~0.1× the base price), in exchange for a modest one-time write premium (often ~1.25×).

The economics are striking. The write premium is a small fraction of the prefix cost; the saving on each reuse is most of it. So the break-even point is well under one reuse — caching pays for itself the first time the prefix is reused, and everything after is nearly free. The catch: the prefix must be stable and identical (caches key on exact prefix match), so put the fixed content first and the variable content last. Get that ordering right and a huge system prompt stops mattering to your bill.

The base request cost, with prices p_in and p_out quoted per million tokens:

Because p_out ≫ p_in, an output token costs several input tokens — so the same total tokens cost more when weighted toward output.

With caching, the reused prefix of T_pre tokens is billed at a read multiple instead of full price. The reuses needed to break even against the write premium are:

Independent of prefix size and price — the ratio cancels. Under 1 reuse means caching pays from the first reuse. The runnable calculator below computes request cost, the cache discount, and the break-even.

Beyond shorter outputs and caching, a handful of levers cut spend without cutting quality:

Order them by leverage, not novelty: measure first, then cut output, then cache, then route. A cheaper model that's right 95% of the time on easy traffic, with the expensive model as fallback, can slash the bill while barely moving quality — that's serving economics meeting cost engineering. And context engineering — sending less, more relevant input — compounds with all of it.

The classic mistake is optimizing without measuring. Cost is dominated by a few request types; guess wrong and you'll tune the cheap ones. Instrument token counts per endpoint first — you'll usually find one verbose feature is most of the bill. Another trap: breaking the cache by putting variable content (a timestamp, a user id) at the front of the prompt, which changes the prefix and defeats caching for everyone downstream. Keep the fixed part first.

Also beware false economy: a cheaper model that fails and triggers retries, escalations, or human review can cost more end-to-end than the pricier model that just gets it right. Cost per successful outcome is the real metric, not cost per call. And reasoning models add a wrinkle — their hidden "thinking" tokens are billed as output, so a model that thinks a lot is expensive even when its visible answer is short. The discipline is the same throughout: know the token equation, measure where the tokens go, and pull the biggest lever first.