Vibe Engines Learn
Vibe Engines
Handbooks  /  Context Engineering
AI Engineering~13 min readIntermediate
Deep Dive

Context Engineering:
the right tokens, at the right time.

The best prompt in the world loses to a well-curated context window. Context engineering is the discipline that replaced prompt-tweaking: deciding what the model sees at every step — instructions, tools, retrieved knowledge, history, memory — and, just as deliberately, what it doesn't. Here's the whole craft in one sitting.

01

From prompts to context

For a couple of years, getting better output meant wordsmithing the instruction — prompt engineering. That still matters. But the moment you build anything real — a RAG app, a copilot, an agent — the instruction becomes a small slice of what the model actually reads. The rest is assembled by your system: tool definitions, retrieved documents, conversation history, memory, the last tool's output.

Context engineering is the art and science of filling the context window with just the right information at each step. The unit of optimization stops being the sentence and becomes the whole window — and in an agent, that window is rebuilt every single turn.

→ The one-line definition

Prompt engineering asks "how do I phrase this?" Context engineering asks "what should the model see at this step — and what should it not?"

The term went mainstream in 2025–26 because everyone hit the same wall: models got smart enough that the bottleneck moved from the model's ability to reason to the system's ability to feed it the right material. Garbage in the window, garbage out of the model — no matter how good the model is.

02

What actually fills the window

Before optimizing the context, inventory it. A production request is a stack of segments, each competing for the same budget:

Anatomy of one request's context
SystemSystem prompt — identity, rules, output contract. Small, stable, always present.
ToolsTool definitions — every schema the model might call. Grows fast; mostly unread.
ExamplesFew-shot examples — canonical demonstrations of the behavior you want.
KnowledgeRetrieved content — RAG chunks, files, search results, loaded just-in-time.
HistoryConversation / trajectory — prior turns, tool calls and their results.
MemoryPersistent memory — durable facts about the user or task, written across sessions.
RequestThe current message — the thing the user actually asked, often the smallest part.
Every segment is useful. Every segment steals budget from the others. That tension is the job.

Two practical tools for feeling this tension: the Context Window Visualizer (watch the segments fight for space) and the Context Cost Planner (what that space costs in dollars and latency).

03

Why "just add more context" fails

Million-token windows tempt you to shovel everything in. Resist. Three separate forces punish an overstuffed window:

The three taxes on a fat context

Attention tax

  • Recall degrades as the window fills — context rot
  • Middle content is recalled worst ("lost in the middle")
  • Irrelevant or stale text actively distracts the model

Cost & latency tax

  • Every token is prefill compute you pay for, every call
  • Time-to-first-token grows with input length
  • Long contexts break prompt-cache hits when they churn

The third force is subtler: behavioral drift. A window full of old errors, dead ends, and contradictory snippets teaches the model the wrong pattern for this turn — it imitates the mess. An agent that keeps every failed attempt in view gets worse at the task, not better.

→ Mental model

Treat attention as a budget, not a bucket. Every token you include is a claim that it's worth more than the token you left out. Spend deliberately; audit ruthlessly.

04

The four operations

Nearly every context-engineering technique is one of four moves:

OperationThe moveIn practice
WritePersist information outside the window so it survives without occupying it.Scratchpads and note files, durable memory stores, "write down what you learned" steps between turns.
SelectPull into the window only what this step needs.Retrieval, just-in-time file loading, choosing 3 relevant tools instead of exposing 40.
CompressShrink what must stay.Compaction (summarize the oldest history), truncating tool outputs, distilling documents to the lines that matter.
IsolateSplit work so no single window carries everything.Sub-agents with clean, task-scoped contexts that return only findings; sandboxes holding state outside the model.

Notice what's not on the list: "buy a bigger window." Window size changes how much you can include; the four operations decide what you should.

05

Where it gets hard: agents

A single Q&A call is a one-shot packing problem. An agent is a packing problem that compounds: every loop iteration appends tool results, observations and decisions, and yesterday's clever context is today's rot. Long-horizon tasks force the full toolkit:

Context over an agent's lifetime
Turn 1Small, clean: system prompt + task + tools. The model is sharp.
Turn 12Tool outputs pile up. Trim aggressively: keep conclusions, drop raw dumps.
Turn 30Approaching the budget. Compact: summarize the oldest trajectory into a paragraph; keep recent turns verbatim.
Turn 50+Long-horizon: write durable notes to files/memory, isolate subtasks into sub-agents, reload state just-in-time.

Two patterns do most of the work. Just-in-time retrieval: instead of pre-loading everything the agent might need, give it lightweight references (file paths, doc ids, queries) and tools to load them when needed — the way a human keeps tabs closed until they matter. Structured note-taking: have the agent maintain its own state file — objectives, decisions, open questions — that persists outside the window and gets pulled back after compaction.

This is also where context engineering meets loop engineering: the loop decides when to compact, when to spawn a sub-agent, when to re-read notes. The two disciplines are the software half of what the model's intelligence can't do alone.

06

The craft, in six habits

Day-to-day, good context engineering looks like this:

Habits that survive production
AltitudeWrite the system prompt at the right altitude — firm on outcomes and guardrails, silent on micro-steps. Too rigid breaks on edge cases; too vague invites drift.
Tool dietExpose the minimum viable toolset. Every unused schema is dead weight; overlapping tools confuse the picker. If a human can't say which tool fits, neither can the model.
Canonical examplesA few diverse, canonical few-shots beat a laundry list of edge cases. Examples are pictures worth a thousand rules.
Cache-awareKeep the prefix stable — system prompt and tools first, volatile content last — so prompt caching keeps hitting and your prefill bill collapses.
Fresh over fullPrefer summaries of old state + verbatim recent state over a full verbatim history. Recency deserves fidelity; the past earns a précis.
MeasureEval context changes like code changes. A retrieval tweak or a compaction prompt can move quality more than a model upgrade — and can silently regress it, too.
07

The stack: prompt → context → loop → harness

Context engineering is one layer of a stack that's crystallized around building with models:

LayerQuestion it answersWhere to go deeper
Prompt engineeringHow do I phrase this instruction?The Prompting Handbook
Context engineeringWhat should the window contain at this step?You're here.
Loop engineeringHow does each output feed the next input — and when do we stop?Loop Engineering
Harness engineeringWhat orchestration code runs the loop safely — tools, retries, budgets, evals?The AI Harness Roadmap

They compose: the harness runs the loop, the loop reassembles the context, the context carries the prompt. Master one and the neighbors start mattering immediately — which is exactly why "prompt engineer" quietly became "AI engineer."

Put context engineering to work
Context Window Visualizer — watch the budget fill Context Cost Planner — price the window Challenge — trim to a token budget Lab — the loop that rebuilds context Design ChatGPT — context at planet scale
Frequently asked

Quick answers

What is context engineering?

The discipline of curating everything the model sees in its window — system prompt, tools, examples, retrieved knowledge, history, memory — so each inference step gets exactly what it needs and nothing it doesn't.

How is it different from prompt engineering?

Prompt engineering optimizes one instruction; context engineering manages the whole window over time — a budget allocated across competing sources, reassembled every turn in an agent.

Why doesn't a bigger window solve it?

Attention is finite: recall degrades as the window fills (context rot), middle content gets lost, and every token adds prefill cost and latency. Bigger windows raise the ceiling, not the quality.

What are the four operations?

Write (persist outside the window), Select (retrieve only what's relevant), Compress (summarize and trim to budget), Isolate (split across sub-agents with clean windows).

Context Engineering · AI Engineering · Vibe Engines · 2026
Finished this one? 0 / 49 Handbooks done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

AI Agents & Tools

More Handbooks

Explore more from Vibe Engines

Roadmaps →
System Designs →
AI System Designs →
Algorithms →
Labs →
Paper Breakdowns →