LABS · 19  /  CONTEXT

The Crowded Desk.

Give a model a bigger context window and it should get smarter, right? Bury one crucial fact — the needle — in a wall of filler and find out. Move it around the window, watch it sink in the middle, and watch recall rot as the desk fills. A fuller window is not a smarter one.

Act 1 · Position matters
Move the needle, watch recall shift
The window holds one gold needle fact, a blue question at the end, and a lot of gray filler. Slide the needle from top to bottom. Recall is strong at the edges and weakest in the middle — the U-shaped "lost in the middle" curve.
Needle positiontop
Recall probability
Where in window
Verdict
Same fact, same model — only the position changed, yet recall swings wildly. This is why prompt order is a design decision: put what matters near the top or, better, right next to the question.
Act 2 · The desk fills
Recall rots as you add filler
Now hold the needle in the middle and pour in filler. Attention is a finite budget spread over every token — the more low-signal content you pile on, the less reliably the model attends to the one fact that matters. Watch the curve fall.
Context fill8K tokens
Middle-needle recall
Prefill cost
relative to 8K
Signal : noise
Bigger window, worse recall AND higher cost — the double tax of context rot. A 128K prompt that’s 99% filler is slower, pricier, and less accurate than a tight 4K prompt with just the right facts.
Curate, don't cram
The fix isn't a bigger desk — it's a tidier one. Toggle the three moves of context engineering and watch recall recover: retrieve only what's relevant, reposition it next to the question, and compact the old filler away.
62%
Recall probability
128K
Tokens in window
1%
Signal density
Start with the rotted 128K window and add the moves. Each one raises recall and cuts tokens at once — because the goal was never "fit more", it was "the model reliably sees what matters".

Why more context can mean less intelligence

It feels like a bigger context window should strictly help — more room for more information. But two effects push the other way, and together they are context rot.

First, lost in the middle: models recall content at the start and end of a long context far better than content buried in the center. Retrieval accuracy traces a U by position — strong at the edges, weakest in the middle. Where a fact sits changes whether the model uses it at all.

Second, attention dilution: attention is a finite budget spread across every token in the window. Pour in filler and each token — including the one that matters — gets a thinner slice. Add the model's tendency to be distracted by irrelevant-but-plausible text, and accuracy sags as the window fills, regardless of position.

The counterintuitive law

A fuller window is not a smarter window. Bigger context raises the ceiling on what you can include and costs more prefill for the privilege — but the middle is still weakest and noise still dilutes. Capacity is not quality.

The discipline: context engineering

The answer is to curate the window rather than fill it — the whole point of context engineering. Retrieve only the relevant chunks instead of dumping everything. Position the most important content near the question, exploiting the strong edge rather than fighting the weak middle. Compact old turns into short summaries before they bury the signal. Keep durable facts in a small always-visible core instead of scrolling history.

Every one of those moves does the same thing: raises the signal density of the window. That — not window size — is what actually determines whether the model can find the needle.

Check yourself

Four questions. Don't let them rot.

Finished this one? 0 / 22 Labs done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

More Labs