Mamba,
explained.

The Transformer's power comes from attention: at every layer, each token looks at every other token to decide what matters. That all-pairs comparison is why it's so good — and why it's so costly. For a sequence of length N, attention does on the order of N² work and memory. Double the context and the cost roughly quadruples.

For a paragraph, fine. For a whole book, a genome, an hour of audio, or a million-token context, that quadratic wall bites hard. Worse, generating text with a Transformer means caching the keys and values of every past token — memory that grows without bound as the output gets longer. The field wanted attention's quality without attention's scaling. Many tried; most produced models that were fast but noticeably dumber.

Mamba builds on state-space models (SSMs), a formalism borrowed from control theory. An SSM processes a sequence by carrying a hidden state — a compact summary of everything seen so far — and updating it with simple linear rules at each step, much like an RNN or a physical system evolving over time.

The crucial difference from attention: an SSM doesn't keep every past token around. It compresses the history into a fixed-size state. That's the whole efficiency story — a constant-size memory instead of a cache that grows with the sequence, and per-step cost that doesn't blow up with length. The problem, historically, was making that compressed state expressive enough to rival attention.

A recurrence is great for cheap, one-step-at-a-time generation but painful to train — you can't parallelize across a sequence. Attention and convolutions parallelize beautifully for training but pay for it elsewhere. The prior SSM work (S4) found a remarkable trick: the same linear state-space model can be computed two ways.

During training, unroll the recurrence into a giant convolution and run the whole sequence in parallel on a GPU. During generation, run it as a step-by-step recurrence with a constant-size state. Same math, two modes — parallel where you want speed, recurrent where you want cheap inference. That duality is what makes SSMs practical rather than just elegant.

Earlier SSMs had a fatal flaw on language: their update rules were fixed — the same for every token, independent of content. That works for smooth signals like audio, but language needs judgment: skip the filler word "the," but hang on tightly to the subject you'll need ten sentences later. A model that treats every token identically can't do that, and that rigidity is why SSMs kept losing to attention on text.

Mamba's core contribution is selectivity: it makes the SSM's parameters input-dependent. The rules for what to remember, what to forget, and how much a token updates the state now change based on the token itself. Suddenly the model can choose to store important information and let irrelevant tokens flow past — content-aware memory. This one change is what finally closed the quality gap with Transformers on language.

Selectivity came with a catch. Making the parameters input-dependent broke the neat convolutional trick that made older SSMs fast — you can no longer precompute one fixed kernel. Mamba's answer is a hardware-aware parallel algorithm: a carefully written GPU scan that computes the selective recurrence efficiently while keeping the working state in fast on-chip memory (SRAM) and avoiding costly trips to slower GPU memory.

This is a recurring theme in modern deep learning — echoing FlashAttention. The algorithm on paper isn't enough; you have to co-design it with how GPUs actually move data. Mamba's speed comes as much from this systems engineering as from the math.

Mamba matched or beat Transformers of comparable size on language modeling, and did so with linear scaling in sequence length and up to several times faster inference. It also shone on domains where sequences are naturally very long — audio and genomics — where the quadratic cost of attention is especially punishing.

That combination — competitive quality and linear efficiency — is exactly what a decade of attention-alternatives had failed to deliver together, which is why Mamba drew so much attention (so to speak).

Compressing all history into a fixed-size state is efficient but lossy. Attention has perfect random access — it can reach back and read any earlier token exactly. Mamba only has whatever survived in its compressed state, so on tasks that need precise recall of a specific distant token (certain copying or in-context lookup tasks), pure SSMs can lag attention.

In practice, the strongest systems often go hybrid — interleaving Mamba-style SSM layers with a few attention layers to get linear efficiency plus occasional exact recall. And as a newer architecture, it lacks the Transformer's years of tooling, tricks, and ecosystem. It's a serious challenger, not (yet) a wholesale replacement.

Mamba reopened a question many had considered closed: does the future of sequence modeling have to be attention? By showing an SSM could match Transformers on language while scaling linearly, it made non-attention architectures a live, credible research direction again — and pushed long-context modeling forward where quadratic cost had stalled it.

Its influence is already visible in hybrid models that blend SSM and attention layers, and in a renewed hunt for efficient architectures for very long sequences. Whether or not Mamba itself becomes dominant, it re-legitimized the search for something beyond attention — and that shift in the field's imagination is its real contribution.