Context rot is the decline in a language model's accuracy and recall as its context window fills with more content — especially irrelevant or low-signal content. Attention is a finite budget spread across all tokens, so the more you cram in, the less reliably the model attends to any particular fact. A fuller window is not a smarter one.

What is the lost-in-the-middle effect?

Models recall information placed at the beginning and end of a long context far better than information buried in the middle. Retrieval quality follows a U-shaped curve by position: strong at the edges, weakest in the center. It means where you put a fact in the prompt materially changes whether the model uses it.

Doesn't a bigger context window solve this?

No. A larger window raises the ceiling on how much you can include, but the middle is still recalled worst, irrelevant tokens still dilute attention, and every extra token costs prefill latency and money. Bigger buys capacity, not quality — which is why context engineering focuses on putting the right things in, not the most things.

How do you fight context rot?

By curating the window instead of filling it: retrieve only the relevant chunks (and place the most important near the question), compact or summarize old turns before they bury the signal, and keep durable facts in a small always-visible core rather than scrolling history. This is the discipline of context engineering.

LABS · 19 / CONTEXT

The Crowded Desk.

Give a model a bigger context window and it should get smarter, right? Bury one crucial fact — the needle — in a wall of filler and find out. Move it around the window, watch it sink in the middle, and watch recall rot as the desk fills. A fuller window is not a smarter one.

Act 1 · Position matters

Move the needle, watch recall shift

The window holds one gold needle fact, a blue question at the end, and a lot of gray filler. Slide the needle from top to bottom. Recall is strong at the edges and weakest in the middle — the U-shaped "lost in the middle" curve.

Needle positiontop

—

Recall probability

—

Where in window

—

Verdict

Same fact, same model — only the position changed, yet recall swings wildly. This is why prompt order is a design decision: put what matters near the top or, better, right next to the question.

Act 2 · The desk fills

Recall rots as you add filler

Now hold the needle in the middle and pour in filler. Attention is a finite budget spread over every token — the more low-signal content you pile on, the less reliably the model attends to the one fact that matters. Watch the curve fall.

Context fill8K tokens

—

Middle-needle recall

—

Prefill cost

relative to 8K

—

Signal : noise

Bigger window, worse recall AND higher cost — the double tax of context rot. A 128K prompt that’s 99% filler is slower, pricier, and less accurate than a tight 4K prompt with just the right facts.

Act 3 · Context engineering

Curate, don't cram

The fix isn't a bigger desk — it's a tidier one. Toggle the three moves of context engineering and watch recall recover: retrieve only what's relevant, reposition it next to the question, and compact the old filler away.

Retrieve relevant only Put needle near question Compact old turns

62%

Recall probability

128K

Tokens in window

Signal density

Start with the rotted 128K window and add the moves. Each one raises recall and cuts tokens at once — because the goal was never "fit more", it was "the model reliably sees what matters".

Why more context can mean less intelligence

It feels like a bigger context window should strictly help — more room for more information. But two effects push the other way, and together they are context rot.

First, lost in the middle: models recall content at the start and end of a long context far better than content buried in the center. Retrieval accuracy traces a U by position — strong at the edges, weakest in the middle. Where a fact sits changes whether the model uses it at all.

Second, attention dilution: attention is a finite budget spread across every token in the window. Pour in filler and each token — including the one that matters — gets a thinner slice. Add the model's tendency to be distracted by irrelevant-but-plausible text, and accuracy sags as the window fills, regardless of position.

The counterintuitive law

A fuller window is not a smarter window. Bigger context raises the ceiling on what you can include and costs more prefill for the privilege — but the middle is still weakest and noise still dilutes. Capacity is not quality.

The discipline: context engineering

The answer is to curate the window rather than fill it — the whole point of context engineering. Retrieve only the relevant chunks instead of dumping everything. Position the most important content near the question, exploiting the strong edge rather than fighting the weak middle. Compact old turns into short summaries before they bury the signal. Keep durable facts in a small always-visible core instead of scrolling history.

Every one of those moves does the same thing: raises the signal density of the window. That — not window size — is what actually determines whether the model can find the needle.

Check yourself

Four questions. Don't let them rot.

Finished this one? 0 / 61 Labs done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

AI Agents & Tools

The Crowded Desk.

Why more context can mean less intelligence

The discipline: context engineering

Check yourself

Explore the topic

More Labs

Explore more from Vibe Engines

Why more context can mean less intelligence

The discipline: context engineering

Check yourself

Keep going

Context Engineering →

Master Context Engineering →

Context Window Visualizer →

MemGPT →

Explore the topic

More Labs

Explore more from Vibe Engines