Mamba,
explained.

Rungs 1 to 3 all keep attention and change its bookkeeping. This rung changes the primitive. A state-space model carries a fixed-size state forward through the sequence, updating it at each token — which is recurrence, and therefore linear in sequence length with a per-token inference cost that does not grow at all. Against a transformer that is a different cost structure, not a cheaper one: no quadratic term, and no key-value cache that swells with every token generated.

Earlier state-space models had that cost structure and were not competitive on language, and this paper's diagnosis of why is the actual contribution. Their state update was the same at every position, which makes the recurrence expressible as a long convolution and therefore fast to train — but it means the model cannot decide to ignore a token or to hold on to one. It compresses everything at a fixed rate. Attention gets selectivity for free, because a query chooses what to look at.

The change is to make the state update a function of the current input: the model learns what to admit into the state and what to forget, per token. That breaks the convolution trick, since the operation is no longer the same at every position, so the paper pays for selectivity with a hardware-aware parallel scan that keeps the state in fast on-chip memory and never writes the full sequence of states out. Stated in the ladder's units: linear time and a constant-size inference state, with the content-dependence that made attention work in the first place.

The Transformer's power comes from attention: at every layer, each token looks at every other token to decide what matters. That all-pairs comparison is why it's so good — and why it's so costly. For a sequence of length N, attention does on the order of N² work and memory. Double the context and the cost roughly quadruples.

For a paragraph, fine. For a whole book, a genome, an hour of audio, or a million-token context, that quadratic wall bites hard. Worse, generating text with a Transformer means caching the keys and values of every past token — memory that grows without bound as the output gets longer. The field wanted attention's quality without attention's scaling. Many tried; most produced models that were fast but noticeably dumber.

Mamba builds on state-space models (SSMs), a formalism borrowed from control theory. An SSM processes a sequence by carrying a hidden state — a compact summary of everything seen so far — and updating it with simple linear rules at each step, much like an RNN or a physical system evolving over time.

The crucial difference from attention: an SSM doesn't keep every past token around. It compresses the history into a fixed-size state. That's the whole efficiency story — a constant-size memory instead of a cache that grows with the sequence, and per-step cost that doesn't blow up with length. The problem, historically, was making that compressed state expressive enough to rival attention.

The update rule is a plain linear recurrence — carry a state h, fold in each input, read out an output:

Because the state is fixed-size and each step is O(1), the whole sequence is O(N) compute and O(1) memory — against attention's O(N²) and a KV cache that grows to O(N):

The state never grows with the sequence — that flat line is the entire pitch. The runnable version below computes it, and proves the recurrence equals a convolution (the trick that makes training parallel).

A recurrence is great for cheap, one-step-at-a-time generation but painful to train — you can't parallelize across a sequence. Attention and convolutions parallelize beautifully for training but pay for it elsewhere. The prior SSM work (S4) found a remarkable trick: the same linear state-space model can be computed two ways.

During training, unroll the recurrence into a giant convolution and run the whole sequence in parallel on a GPU. During generation, run it as a step-by-step recurrence with a constant-size state. Same math, two modes — parallel where you want speed, recurrent where you want cheap inference. That duality is what makes SSMs practical rather than just elegant.

Earlier SSMs had a fatal flaw on language: their update rules were fixed — the same for every token, independent of content. That works for smooth signals like audio, but language needs judgment: skip the filler word "the," but hang on tightly to the subject you'll need ten sentences later. A model that treats every token identically can't do that, and that rigidity is why SSMs kept losing to attention on text.

Mamba's core contribution is selectivity: it makes the SSM's parameters input-dependent. The rules for what to remember, what to forget, and how much a token updates the state now change based on the token itself. Suddenly the model can choose to store important information and let irrelevant tokens flow past — content-aware memory. This one change is what finally closed the quality gap with Transformers on language.

Selectivity came with a catch. Making the parameters input-dependent broke the neat convolutional trick that made older SSMs fast — you can no longer precompute one fixed kernel. Mamba's answer is a hardware-aware parallel algorithm: a carefully written GPU scan that computes the selective recurrence efficiently while keeping the working state in fast on-chip memory (SRAM) and avoiding costly trips to slower GPU memory.

This is a recurring theme in modern deep learning — echoing FlashAttention. The algorithm on paper isn't enough; you have to co-design it with how GPUs actually move data. Mamba's speed comes as much from this systems engineering as from the math.

Mamba matched or beat Transformers of comparable size on language modeling, and did so with linear scaling in sequence length and up to several times faster inference. It also shone on domains where sequences are naturally very long — audio and genomics — where the quadratic cost of attention is especially punishing.

That combination — competitive quality and linear efficiency — is exactly what a decade of attention-alternatives had failed to deliver together, which is why Mamba drew so much attention (so to speak).

Compressing all history into a fixed-size state is efficient but lossy. Attention has perfect random access — it can reach back and read any earlier token exactly. Mamba only has whatever survived in its compressed state, so on tasks that need precise recall of a specific distant token (certain copying or in-context lookup tasks), pure SSMs can lag attention.

In practice, the strongest systems often go hybrid — interleaving Mamba-style SSM layers with a few attention layers to get linear efficiency plus occasional exact recall. And as a newer architecture, it lacks the Transformer's years of tooling, tricks, and ecosystem. It's a serious challenger, not (yet) a wholesale replacement.

Mamba reopened a question many had considered closed: does the future of sequence modeling have to be attention? By showing an SSM could match Transformers on language while scaling linearly, it made non-attention architectures a live, credible research direction again — and pushed long-context modeling forward where quadratic cost had stalled it.

Its influence is already visible in hybrid models that blend SSM and attention layers, and in a renewed hunt for efficient architectures for very long sequences. Whether or not Mamba itself becomes dominant, it re-legitimized the search for something beyond attention — and that shift in the field's imagination is its real contribution.

The structural limit is not a bug and cannot be tuned away: a fixed-size state cannot losslessly retain an arbitrary amount of detail. A transformer's key-value cache is, in effect, a perfect transcript — costly, and perfect. A recurrent state is a summary. So on tasks that require verbatim recall of something specific from far back — copying, retrieving an exact string, the needle-in-a-haystack family — attention has an advantage that follows from information capacity rather than from training. This is measured, repeatedly, by others.

The evidence base at publication was also mid-scale. Claims about a new primitive at small scale have a poor historical record, and the honest position was that the comparison at frontier scale had not been run.

How it held up: as a component, decisively; as a replacement, not yet. What actually shipped is hybrids — mostly state-space layers with a minority of full attention layers interleaved — and the reason is exactly the admission above: the attention layers supply the exact recall the recurrent state cannot, and the recurrent layers supply the cheap long-range mixing attention cannot afford. Rungs 5 and 6 above are two further attempts at the same gap, which is the clearest sign that the gap is real and open.

Answer each one out loud before you open it. Recalling an answer is what makes it stick; recognising one you have just read is not. Your score is kept on this device.

Attention costs quadratic in sequence length. Six attempts to escape that, ending with two 2026 papers that give the model a memory instead.

1. 1Transformer-XL2019
2. 2Longformer2020
3. 3Ring Attention2023
4. 4Mamba2023
5. 5Titans2025
6. 6Engram2026

All 110 papers, in 19 ladders →