Engram,
explained.

Softmax attention answers a query by spreading a fixed budget of probability across every token it can see. That's fine for a short context. But the needle-in-a-haystack test — hide one crucial fact in a long document and ask for it — exposes the flaw: as the haystack grows, the needle's slice of the attention budget shrinks, and it can get washed out by thousands of irrelevant tokens.

Two costs in one: you pay O(n) to scan everything, and the answer quality drops with length. Engram attacks the second one head-on.

Engram bolts a key-addressed memory onto the model — an associative store. To remember a fact, write it under a key. To recall it, present the key and read the value back in constant time. No scan, no dilution: the lookup cost is O(1), completely independent of how much context has streamed by.

The store is the same associative-memory idea behind MemGPT and Titans — an explicit place to put facts so the attention window doesn't have to hold everything itself.

The "conditional" in conditional memory is the second half of the trick. The model doesn't hit the memory on every token — most tokens only need their immediate neighbourhood. A lightweight gate predicts whether the current position actually needs a long-range recall, and only then does a lookup happen.

So the memory adds a capability — perfect long-range recall — without adding a cost to the vast majority of positions that never needed it.

Here's the payoff. Because a keyed lookup retrieves by identity, not by out-competing every other token, the needle-in-a-haystack score stops falling off with length. DeepSeek reports NIAH climbing from about 84.2 to 97 with Engram — the recall that softmax attention was quietly losing on long inputs, restored.

And it's why Engram underpins DeepSeek-V4: V4's hybrid attention makes the long context cheap, and Engram makes sure the model can still find things in it. Cheap plus accurate is what a usable million-token window actually requires.

Engram is part of a clear shift: stop asking a fixed attention window to be the model's whole memory, and give it an explicit, addressable place to store and fetch facts. It's the architectural cousin of the agent-memory patterns builders already use — MemGPT's tiered paging, Titans' surprise-gated writes, and the agent-memory playbook — but baked into the model instead of bolted on by a harness.

The broader lesson is that "long context" is really two problems wearing one name: the cost of a big window and the reliability of recall inside it. The 2026 frontier is solving them separately — sparse/linear attention for the first, addressable memory for the second — and Engram is the cleanest statement yet of the second half.