Context Budget & Cost Planner

Every turn of a conversation re-sends the whole context, so the bill depends on what sits in that context far more than on the user's question. This planner adds up the system prompt, tool definitions, retrieved documents and growing chat history, shows how the context window fills, and prices one conversation, a thousand conversations, a day and a month on one of three price tiers.

What it computes

Call the fixed part of every request B: system prompt + tool definitions + RAG context. Turn i of a t-turn conversation sends B, the i user messages so far and the i − 1 earlier replies:

input on turn i    = B + i × u + (i − 1) × r
total input        = t × B + u × t(t + 1)/2 + r × t(t − 1)/2
total output       = t × r
peak context       = B + t × (u + r)
cost/conversation  = input/1M × P_in + output/1M × P_out
cost/day           = cost/conversation × requests per day
cost/month         = cost/day × 30

• u: user tokens per turn. r: output tokens per turn; each reply becomes history for the next turn.
• t: turns in one conversation. One "request" in the daily volume is one whole conversation.
• P_in, P_out: the selected tier's price per million tokens. The built-in tiers are $3/$15, $1/$5 and $0.25/$1.25.

The colored bar splits the peak context into system prompt, tools, RAG and history, where history is t × (u + r). Token counts are entered directly; the tool does not tokenize text.

A worked example

The defaults: a 900-token system prompt, 1,200 tokens of tool definitions, 3,000 tokens of RAG context, 150 user and 400 output tokens per turn, 8 turns, the $3/$15 tier and 10,000 conversations a day.

1. Fixed context: B = 900 + 1,200 + 3,000 = 5,100 tokens, sent on all 8 turns, so 40,800 tokens.
2. User messages: 150 × 36 = 5,400. Replies re-sent as history: 400 × 28 = 11,200.
3. Total input: 57,400 tokens. Output: 8 × 400 = 3,200.
4. Peak context: 5,100 + 8 × 550 = 9,500 tokens, split 9.5% system, 12.6% tools, 31.6% RAG and 46.3% history.
5. Cost per conversation: 57,400/1M × $3 = $0.1722 plus 3,200/1M × $15 = $0.0480 = $0.2202.
6. Scale: $220.20 per 1k conversations, $2,202.00 a day and $66,060 a month.

The same conversation on each tier:

Note where the input goes: the fixed context accounts for 40,800 of the 57,400 input tokens, 71%, even though history is the largest slice of the peak window.

Assumptions and limits

• Uniform turns. Every user message and reply has the same length. Real conversations vary, and one long pasted document changes everything after it.
• Full history, every turn. Nothing is summarized, windowed or dropped, and RAG context is retrieved once and kept for the whole conversation.
• No caching. Every input token is billed at the full input price. Providers that cache a repeated prefix bill it at a discount, which mostly hits the fixed context.
• Three fixed tiers. The prices are illustrative points, not a specific vendor's list. The model's context window is not checked against the peak.
• A 30-day month at constant daily volume.

How to use the result

• Attack the slice that repeats most. The fixed context is billed once per turn, so in a long conversation it is usually the cheapest place to win: a shorter system prompt, leaner tool schemas, fewer RAG chunks or a cached prefix.
• Cap history when turns grow. History cost rises with the square of the turn count, so beyond a few turns a rolling window or a running summary pays for itself.
• Test one lever at a time. Halving RAG context to 1,500 tokens brings the default to $0.1842, 16% less. Comparing levers this way stops you from optimizing the wrong slice.
• Keep peak context well inside the window. Leave room for the reply and for the longest conversations, not the average one.
• Route by difficulty. Each tier down is 3 to 4 times cheaper here, so sending routine turns to a smaller model can beat any prompt trimming.

When to use it, and when not

Use it while a chat feature is still on paper: to set a token budget for each slice, choose a price tier, decide when history needs a window or a summary, and turn expected traffic into a monthly figure before launch.

It models one model call per turn. Agents that make several calls inside a turn, each re-sending the growing context, are better priced with an agent-loop estimator. Once you have traffic, sum the token counts your provider returns with each response; they include caching, real message lengths and conversations that end early.

Trade-offs and limits

The window saving grows with length because it removes the quadratic term: past the window, every turn costs the same. The price is memory, since anything older than the window is gone unless a summary carries it forward.

Common mistakes

• Entering API calls as requests. The daily volume counts whole conversations. 10,000 calls a day from 8-turn chats are 1,250 conversations, or $275.25 a day; entering 10,000 overstates the bill eightfold.
• Counting tokens, not dollars. Output is 3,200 of the 60,600 tokens in the default, about 5%, yet at five times the input price it is $0.0480, 22% of the cost.
• Putting changing content first. Caching reuses a prefix that is identical from the start. Keep the system prompt and tool definitions first and the retrieved chunks, which change per conversation, after them.
• Sizing the window for the average conversation. The peak grows by 550 tokens a turn here. The longest conversations you allow, not the typical one, must fit.
• Retrieving on every turn without dropping old chunks. Each new set of chunks becomes history for every later turn, the 2.1× case in the table above.

Check yourself