Concept circuits,
explained.

Rungs 1 to 3 build the machine. This rung asks what is inside it, and the first finding is why that question is hard: a single neuron does not mean one thing. Inspect the activations of an individual unit in a large model and it fires for an unrelated-looking collection of inputs. The obvious interpretation — that the model's concepts are simply not human-legible — turns out to be the wrong one.

The better account is superposition. A model has some number of dimensions and wants to represent far more features than that. If only a few features are active at once — which is true of nearly everything language is about — it can pack many features into directions that are merely nearly orthogonal rather than exactly so, accepting a small amount of interference in exchange for representing far more than its dimension count allows. Polysemantic neurons are then not a failure of interpretability; they are the expected consequence of an efficient code, and the unit of meaning is a direction in activation space, not a neuron.

That reframing supplies the method. To read the model you first have to unpack the superposition — learn an overcomplete dictionary of directions that reconstructs the activations while keeping very few of them active at a time, which recovers features that are individually far more interpretable than any neuron. With features in hand you can then ask the causal question the ladder has been building toward: trace which features fed which, through which layers, to produce a specific output, and read the circuit rather than the weights.

The first thing you learn probing a real network is that individual neurons are polysemantic — a single unit lights up for a jumble of unrelated concepts. It's tempting to read that as the model being messy. It isn't. The model simply isn't storing "one concept per neuron." It's storing concepts as directions in the activation space, and those directions are spread across many neurons at once.

So a neuron isn't a concept — it's one coordinate that many concept-directions happen to use. Ask "what does neuron 4,182 mean?" and you get nonsense; ask "which directions does the model use, and what does each mean?" and you get features. The reason there are more features than neurons is the mechanism worth understanding.

A space with d dimensions holds only d perfectly orthogonal directions. But it holds vastly more directions that are almost orthogonal — and "almost" is good enough. Superposition is the model exploiting this: give each feature its own direction, let those directions be near-orthogonal, and accept a little interference between them.

Three unit vectors in 2-D can't be orthogonal, but at 120° apart their pairwise dot is only −0.5 — bounded interference, and three features in a two-dimensional home.

If superposed features interfere, why doesn't the model turn to mush? Because features are sparse — only a handful are active on any given input. "This text is legal," "this token is a date," "the topic is grief" don't all fire at once. When just one feature is active, reading it back is clean; the interference terms are dormant because the features they'd collide with aren't switched on.

So superposition is a bet on sparsity: pack in many features cheaply, and rely on the fact that they rarely co-activate, so the rare collisions cost little. That's exactly why embedding-style geometry can carry so much meaning in modest dimensions — and why sparse-feature methods (sparse autoencoders) can pull the individual features back out of the tangle.

Once you can name features, you can ask the real question: which features produced this output, and how? An attribution graph measures each feature's contribution to a target — how strongly the output direction reads each active feature direction — and chains those contributions across layers into a circuit: the path of intermediate features the model actually used.

That's the leap from "the model answered Austin" to "it answered Austin because the Texas feature activated the state-capital feature, which drove the Austin output." Recent work adds a self-explanation twist — the model translating its own internal features into natural language — but the backbone is the same: attribution turns a forward pass into a readable causal graph. The runnable below builds the simplest version: contribution as the dot product between a feature and the output direction.

Mechanistic interpretability is what turns a model from a black box into something you can debug. If you can name a model's features and trace the circuit behind a behaviour, you can find why it hallucinates, spot the feature a jailbreak exploits, or catch it leaning on a spurious cue — and sometimes steer it by editing the feature directly. That's a different kind of control than prompting: surgery, not persuasion.

Superposition is the reason it's hard (features hide across neurons) and the reason it's possible (they're still linear directions you can recover). This breakdown is the foundation; the engineering playbook — what you can actually inspect and fix in production — is the natural next step, and it's the same geometry-of-meaning idea that word2vec first made visible.

Concept-circuit work joins two techniques. Sparse dictionary learning recovers features from a model's activations, and attribution graphs trace how those features combine to produce one specific output. Both were developed in public research between 2022 and 2025, chiefly at Anthropic, OpenAI and Google DeepMind.

1. Collect activations. Run the model over a large corpus and record the vector at one site for every token, for example the residual stream after a middle layer, or the input and output of an MLP. Hundreds of millions of token vectors is a normal training set.
2. Train a sparse autoencoder. An encoder maps each d-dimensional activation to m feature activations through a ReLU, with m far larger than d: anywhere from a few times d to millions of features. A linear decoder rebuilds the activation as a weighted sum of m learned directions. The loss is reconstruction error plus a penalty on feature activity, classically an L1 penalty. Decoder directions are held to unit norm, or their norms are folded into the penalty, so training cannot dodge the penalty by shrinking activations and growing directions.
3. Or set the sparsity directly. TopK autoencoders (Gao et al., OpenAI, 2024) keep only the k largest activations per token, and JumpReLU autoencoders (Google DeepMind, 2024) learn a firing threshold per feature. Both aim at L0, the average number of active features per token, instead of tuning a penalty weight.
4. Keep features alive. Some latents stop firing on every input and never recover; these are dead latents. Remedies include re-initializing them toward inputs that are reconstructed badly, as Anthropic did in Towards Monosemanticity, and an auxiliary loss that trains dead latents on the leftover error, as in Gao et al.
5. Label the features. Read each feature's top-activating text, then test its decoder direction: which output tokens it promotes, and what changes when it is added or removed. Automated pipelines have a second model write an explanation and score it by how well it predicts the feature's activations on new text.
6. Switch to transcoders for circuits. An autoencoder reconstructs an activation. A transcoder instead predicts an MLP's output from the MLP's input through the same kind of sparse layer, so its features stand in for the MLP's computation. Anthropic's cross-layer transcoder goes further: each feature reads the residual stream at one layer and writes to the MLP outputs of that layer and every later one.
7. Build a replacement model for one prompt. Swap every MLP for transcoder features, freeze the attention patterns and normalization scales recorded from the real forward pass, and add an error node at each layer and position holding whatever the transcoder missed. On that prompt the replacement reproduces the model's activations exactly, and every remaining interaction is linear.
8. Attribute and prune. In the frozen, linear system, the influence of one node on another is the source's activation times a fixed virtual weight between them. Nodes are input tokens, active features, error nodes and output logits. The graph keeps only the nodes and edges carrying most of the influence on a chosen logit, and researchers group related features into supernodes by hand.
9. Test with interventions. A graph is a hypothesis about the real model, so it is checked in the real model: suppress a feature or inject another and see whether the output moves as the graph predicts. In Anthropic's two-step example, asking for the capital of the state containing Dallas, replacing the Texas features with California features changed the answer from Austin to Sacramento.

The numbers that say whether a dictionary is any good

Five are standard: L0; the fraction of activation variance the reconstruction explains; how much next-token loss rises when the reconstruction is spliced back into the model; the share of dead latents; and an interpretability score from humans or an automated explainer. A dictionary can win on one of these while losing on the others, so a single number never settles which is better.

• x — the model's activation at the chosen site, d dimensions. For a transcoder the target is the MLP's output rather than x itself.
• fi(x) — the activation of feature i: nonnegative, and zero for almost every i on any one token.
• di — feature i's decoder direction, a column of W_dec. This direction is what the model's representation is claimed to contain.
• bdec — a learned offset, subtracted before encoding and added back after.
• λ — the price of a firing feature: how much reconstruction error training will accept to switch one off. Multiplying by ‖di‖ closes the escape of tiny activations on huge directions.
• As→t — the attribution from node s to node t in the replacement model: the source's activation as times the virtual weight ws→t through the residual stream and the frozen attention.

At the extremes: with λ = 0 reconstruction is perfect and the code is dense, so it is no easier to read than the neurons. As λ grows without limit every feature dies and x̂ collapses to b_dec. With m = d there is no room to unpack superposition; as m grows very large, features keep splitting into finer variants, which is why dictionary size is a choice and not a discovery.

The method's evidence comes from a short run of public studies, each scaling up the one before. All five below are named by their own titles; none of their results should be read as coming from a single paper.

The positive case is that unpacking superposition the way the theory says makes features legible, and that interventions on those features change behaviour in the predicted direction. The Golden Gate Bridge feature in Claude 3 Sonnet, clamped high, made the model bring up the bridge in unrelated answers.

The negative results are just as consistent. Splicing a reconstruction back into the model raises next-token loss noticeably even with very large dictionaries, so part of every computation lives in the error. Attribution graphs freeze attention, so they explain what the MLPs did given where attention looked, not why it looked there. Features split, and broad features can be absorbed by narrower ones, which makes a feature's meaning depend on the dictionary. And in 2025 Google DeepMind's interpretability team reported that sparse-autoencoder features did worse than plain linear probes at detecting harmful intent on new data, and deprioritized that line of work.

This rung's own literature is the most explicit about its limits of anything on the site, which is appropriate for a field this young. The dictionary you learn is not guaranteed to be the model's features — it is a decomposition that reconstructs well and is sparse, and those criteria do not uniquely determine an answer. Make the dictionary larger and features split into finer variants, which is either a more precise decomposition or an artefact of the method, and distinguishing those two is an open problem rather than a detail.

Coverage is the second admission. The circuits that have been traced end to end are for narrow, carefully-chosen behaviours, and there is a very large gap between explaining one behaviour in one model and explaining a model. The reconstruction-versus-sparsity trade means some of the activation is always left unexplained, and what hides in that residue is unknown by construction.

Where it stands: as the most active open question about the systems the rest of this site describes, and as the only line of work that could turn claims about what a model "knows" into measurements. Two things are worth holding at once. Superposition is a strong, well-supported explanation for why naive inspection fails, and it was validated by making a prediction that came true: unpack the code as the theory says, and legibility improves. Whether the features recovered are the ones the model uses remains, honestly, contested — and the people doing the work say so more clearly than their summarisers do.

Work each one out before you open it — a number, a prediction or a reason, not a definition. Producing the answer is what makes it stick. Your progress is kept on this device.

The architecture itself: what it replaced, how position is encoded once recurrence is gone, and what it turns out to have learned inside.

1. 1Seq2Seq2014
2. 2Attention Is All You Need2017
3. 3RoPE2021
4. 4Concept Circuits2026

All 110 papers, in 19 ladders →