How is LLM API cost calculated?

LLM APIs bill per token, with prices usually quoted per million tokens. A request's cost is the input tokens times the input price plus the output tokens times the output price, divided by a million. Input tokens are everything you send (system prompt, context, the user's message); output tokens are what the model generates. Because the two are priced separately — and output is typically several times more expensive — you need both counts and both prices to estimate a bill.

Why do output tokens dominate the bill?

Output tokens are usually priced several times higher than input tokens (a common ratio is 5x). So for a given total token count, shifting work from input to output raises the cost sharply, and long generations are the biggest driver of spend in generation-heavy workloads. The practical lever is to constrain output: ask for concise answers, cap max tokens, use structured outputs, and avoid making the model restate the input — trimming output is often the single highest-leverage cost cut.

Does prompt caching save money?

Yes, when you reuse a stable prompt prefix across many calls. Prompt caching stores a prefix so subsequent requests pay a small fraction of the input price for it (a cache read might be a tenth of the base price) in exchange for a modest one-time write premium (often around 1.25x). Because the per-reuse saving is large relative to the write premium, caching typically pays for itself from the very first reuse — making a long, fixed system prompt or document context far cheaper to send repeatedly.

What are the main levers to cut AI cost?

The biggest levers are: shorten outputs (they dominate cost); cache stable prefixes; route easy requests to a cheaper model and reserve the expensive model for hard ones; batch non-urgent work for discounted async pricing; trim context to only what's needed; and avoid redundant calls. Measuring first is essential — instrument token counts per request so you optimize the requests that actually cost money rather than guessing.

Handbooks / AI Cost Engineering

Handbook~15 min readProductionworked math + runnable code

The AI Cost Engineering Handbook

The bill is
just tokens.

A demo that costs pennies becomes a product that costs five figures a month — and teams are shocked, because they never did the arithmetic. But AI spend isn't mysterious: it's a simple sum over tokens, with two prices and a few multipliers. Once you can compute a request's cost by hand, the levers become obvious — and one of them (shorter outputs) is worth more than all the clever ones combined. This handbook turns the invoice into an equation you control.

Cost is a sum Output is the lever Caching pays fast The cost equation Run it yourself The full lever set Pitfalls

Cost is a sum

Every LLM API bills the same way: per token, with prices quoted per million tokens. A request has two token counts — input (everything you send: system prompt, context, the user's message) and output (what the model generates) — and each has its own price. The cost of one request is just input tokens times input price plus output tokens times output price, divided by a million. That's it. No hidden magic; the whole bill is that sum, repeated across every call.

The reason bills surprise people is that they never estimate that sum at scale. A single request costing $0.01 feels free; a million requests a day is $10,000. Cost engineering is nothing more than doing this arithmetic before the invoice, instrumenting the token counts you actually send, and then pulling the levers the equation reveals. And the equation reveals a lopsided one: the two prices are not equal.

The one-sentence version

A request's cost is input-tokens × input-price + output-tokens × output-price (per million) — and because output is priced several times higher, trimming output is the biggest lever.

Output is the lever

Output tokens are typically priced several times higher than input tokens — a 5× ratio is common. That asymmetry changes everything about where cost hides. Two requests with the same total token count can differ 2× in price depending on how the tokens split between input and output. A verbose model that restates the question, thinks out loud, and pads its answer is quietly the most expensive thing in your system.

So the highest-leverage cost cut is almost always the same: make outputs shorter. Ask for concise answers, set a sensible max_tokens, use structured outputs so the model emits only the fields you need, and don't make it echo the input back. This single lever routinely beats the fancy ones, because it attacks the most expensive term in the equation directly. Input is cheaper, so a big fixed context hurts less than a big generation — and the next section shows how to make that fixed context nearly free.

Caching pays fast

Many applications send the same long prefix on every call — a big system prompt, a style guide, a document you keep asking about. Prompt caching stores that prefix so later requests pay only a small fraction of the input price for it (a cache read is often ~0.1× the base price), in exchange for a modest one-time write premium (often ~1.25×).

Why caching pays off almost immediately

Write onceFirst call pays a small premium (~1.25×) to store the prefix.

Read cheapEvery later call pays ~0.1× for the cached prefix instead of full price.

Break-evenThe per-reuse saving dwarfs the premium → pays off in < 1 reuse.

Best forBig stable prefixes reused across many calls (system prompt, docs).

The economics are striking. The write premium is a small fraction of the prefix cost; the saving on each reuse is most of it. So the break-even point is well under one reuse — caching pays for itself the first time the prefix is reused, and everything after is nearly free. The catch: the prefix must be stable and identical (caches key on exact prefix match), so put the fixed content first and the variable content last. Get that ordering right and a huge system prompt stops mattering to your bill.

The cost equation

The base request cost, with prices p_in and p_out quoted per million tokens:

cost = ( T_in·p_in + T_out·p_out ) / 10⁶ with typically p_out ≈ 5·p_in

Because p_out ≫ p_in, an output token costs several input tokens — so the same total tokens cost more when weighted toward output.

With caching, the reused prefix of T_pre tokens is billed at a read multiple instead of full price. The reuses needed to break even against the write premium are:

breakeven = (m_write − 1) / (1 − m_read) = 0.25 / 0.9 ≈ 0.28 reuses

Independent of prefix size and price — the ratio cancels. Under 1 reuse means caching pays from the first reuse. The runnable calculator below computes request cost, the cache discount, and the break-even.

RUN IT YOURSELF

A cost calculator you can edit

Cost engineering is arithmetic you can run. A request costs input-tokens × input-price plus output-tokens × output-price, per million — so 1M input at $3/M is $3, and 1M output at $15/M is $15. Caching bills a reused prefix at a fraction of the input price (~0.1×) for a small one-time write premium (~1.25×), so it pays off in under one reuse, independent of prefix size. And because output is priced ~5× input, the same total tokens cost more when weighted toward output — which is why trimming output is the top lever. Change the token counts, prices, or cache multipliers and watch the bill move.

CPython · WebAssembly

# AI cost engineering is arithmetic on TOKENS. API pricing is per-token (quoted per 1M tokens),
# and INPUT and OUTPUT tokens are priced differently -- output is usually several times pricier.
# request cost = (in_tok*in_price + out_tok*out_price) / 1e6. Prompt CACHING bills a reused
# prompt prefix at a fraction of the input price (cache read ~0.1x) for a one-time write premium
# (~1.25x), so a stable prefix reused across calls pays for itself almost immediately.

def request_cost(in_tok, out_tok, in_price, out_price):
    return (in_tok * in_price + out_tok * out_price) / 1e6          # prices quoted per 1M tokens

def cached_input_cost(cached_tok, fresh_tok, in_price, cache_read_mult=0.1):
    return (cached_tok * in_price * cache_read_mult + fresh_tok * in_price) / 1e6

def cache_breakeven(prefix_tok, in_price, cache_write_mult=1.25, cache_read_mult=0.1):
    write_premium = prefix_tok * in_price * (cache_write_mult - 1)  # one-time extra to write the cache
    saving_per_reuse = prefix_tok * in_price * (1 - cache_read_mult)  # saved each reuse vs full price
    return write_premium / saving_per_reuse                          # reuses needed to break even

print(f"1M input tokens @ $3/M: ${request_cost(1000000, 0, 3.0, 15.0):.2f}")
print(f"1M output tokens @ $15/M: ${request_cost(0, 1000000, 3.0, 15.0):.2f}")
print(f"1000-tok prefix: cached ${cached_input_cost(1000,0,3.0):.6f} vs fresh ${request_cost(1000,0,3.0,0.0):.6f}")
print(f"cache break-even: {cache_breakeven(10000, 3.0):.3f} reuses  (pays from the first reuse)")
print(f"same 2000 tokens: output-heavy ${request_cost(500,1500,3,15):.4f} > input-heavy ${request_cost(1500,500,3,15):.4f}")

The full lever set

Beyond shorter outputs and caching, a handful of levers cut spend without cutting quality:

Lever	How it saves
Shorten outputs	Attacks the most expensive term directly — concise answers, max_tokens caps, structured outputs.
Prompt caching	Reused stable prefixes billed at ~0.1× — pays off from the first reuse.
Model routing	Send easy requests to a cheaper model; reserve the flagship for hard ones (a cheap-smart cascade).
Batching	Async/batch APIs offer large discounts for non-urgent work.
Context trimming	Send only the context you need — retrieval beats stuffing the whole corpus.
Dedup & memoize	Cache full responses for identical requests; don't pay twice for the same answer.

Order them by leverage, not novelty: measure first, then cut output, then cache, then route. A cheaper model that's right 95% of the time on easy traffic, with the expensive model as fallback, can slash the bill while barely moving quality — that's serving economics meeting cost engineering. And context engineering — sending less, more relevant input — compounds with all of it.

Pitfalls

The classic mistake is optimizing without measuring. Cost is dominated by a few request types; guess wrong and you'll tune the cheap ones. Instrument token counts per endpoint first — you'll usually find one verbose feature is most of the bill. Another trap: breaking the cache by putting variable content (a timestamp, a user id) at the front of the prompt, which changes the prefix and defeats caching for everyone downstream. Keep the fixed part first.

Also beware false economy: a cheaper model that fails and triggers retries, escalations, or human review can cost more end-to-end than the pricier model that just gets it right. Cost per successful outcome is the real metric, not cost per call. And reasoning models add a wrinkle — their hidden "thinking" tokens are billed as output, so a model that thinks a lot is expensive even when its visible answer is short. The discipline is the same throughout: know the token equation, measure where the tokens go, and pull the biggest lever first.

Worth knowing

Reasoning models bill their internal chain-of-thought as output tokens — often the majority of a request's cost — so "the answer was short" doesn't mean "the request was cheap." Budget thinking, and measure output tokens including hidden reasoning.

Frequently asked

Quick answers

How is LLM cost calculated?

Per token: input-tokens × input-price + output-tokens × output-price, with prices quoted per million tokens.

Why does output dominate?

Output is priced several times higher than input (often ~5×), so long generations drive the bill — trimming output is the top lever.

Does caching save money?

Yes — a reused stable prefix is billed at ~0.1× for a small write premium, so caching pays off from the first reuse.

What's the biggest lever?

Shorter outputs, then caching, then routing easy requests to a cheaper model — but measure first so you cut what actually costs.

Finished this one? 0 / 139 Handbooks done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

LLM Engineering

More Handbooks

Cite this page

Reference it in your work, paper or an AI's context window.

APASingh, S. (2026). The AI Cost Engineering Handbook. Vibe Engines. https://vibeengines.com/handbook/ai-cost-engineering

MLA

Singh, Saurabh. “The AI Cost Engineering Handbook.” Vibe Engines, 2026, vibeengines.com/handbook/ai-cost-engineering.

BibTeX

@online{vibeengines-ai-cost-engineering,
  author       = {Singh, Saurabh},
  title        = {The AI Cost Engineering Handbook},
  year         = {2026},
  organization = {Vibe Engines},
  url          = {https://vibeengines.com/handbook/ai-cost-engineering}
}

The bill isjust tokens.

Cost is a sum

Output is the lever

Caching pays fast

The cost equation

A cost calculator you can edit

The full lever set

Pitfalls

Quick answers

How is LLM cost calculated?

Why does output dominate?

Does caching save money?

What's the biggest lever?

Go deeper

LLM Serving →

Context Engineering →

Speculative Decoding →

Explore the topic

More Handbooks

Explore more from Vibe Engines

Get the next one in your inbox.

Cite this page

The bill is
just tokens.