Context Engineering:
the right tokens, at the right time.

For a couple of years, getting better output meant wordsmithing the instruction — prompt engineering. That still matters. But the moment you build anything real — a RAG app, a copilot, an agent — the instruction becomes a small slice of what the model actually reads. The rest is assembled by your system: tool definitions, retrieved documents, conversation history, memory, the last tool's output.

Context engineering is the art and science of filling the context window with just the right information at each step. The unit of optimization stops being the sentence and becomes the whole window — and in an agent, that window is rebuilt every single turn.

The term went mainstream in 2025–26 because everyone hit the same wall: models got smart enough that the bottleneck moved from the model's ability to reason to the system's ability to feed it the right material. Garbage in the window, garbage out of the model — no matter how good the model is.

Before optimizing the context, inventory it. A production request is a stack of segments, each competing for the same budget:

Two practical tools for feeling this tension: the Context Window Visualizer (watch the segments fight for space) and the Context Cost Planner (what that space costs in dollars and latency).

Million-token windows tempt you to shovel everything in. Resist. Three separate forces punish an overstuffed window:

The third force is subtler: behavioral drift. A window full of old errors, dead ends, and contradictory snippets teaches the model the wrong pattern for this turn — it imitates the mess. An agent that keeps every failed attempt in view gets worse at the task, not better.

The cost tax is quadratic — and that's the surprise

The cost tax isn't linear, it compounds. An agent that appends every turn without pruning holds roughly k·s tokens on turn k (s = tokens added per turn). You re-pay prefill on the whole window every turn, so the total input processed over K turns is a sum:

Now cap the window with compaction — summarize old history so the context never exceeds, say, C = 4000 tokens. The total goes linear:

The longer the task runs, the wider this gap opens — quadratic vs linear.

Nearly every context-engineering technique is one of four moves:

Notice what's not on the list: "buy a bigger window." Window size changes how much you can include; the four operations decide what you should.

A single Q&A call is a one-shot packing problem. An agent is a packing problem that compounds: every loop iteration appends tool results, observations and decisions, and yesterday's clever context is today's rot. Long-horizon tasks force the full toolkit:

Two patterns do most of the work. Just-in-time retrieval: instead of pre-loading everything the agent might need, give it lightweight references (file paths, doc ids, queries) and tools to load them when needed — the way a human keeps tabs closed until they matter. Structured note-taking: have the agent maintain its own state file — objectives, decisions, open questions — that persists outside the window and gets pulled back after compaction.

This is also where context engineering meets loop engineering: the loop decides when to compact, when to spawn a sub-agent, when to re-read notes. The two disciplines are the software half of what the model's intelligence can't do alone.

Day-to-day, good context engineering looks like this:

Context engineering is one layer of a stack that's crystallized around building with models:

They compose: the harness runs the loop, the loop reassembles the context, the context carries the prompt. Master one and the neighbors start mattering immediately — which is exactly why "prompt engineer" quietly became "AI engineer."