Mechanistic
interpretability.

Reframe the goal. You're not trying to read the model's "mind"; you're trying to answer engineering questions: Why did it hallucinate that citation? Which feature does this jailbreak flip? Is it using the demographic field it's not supposed to? Mechanistic interpretability answers those by reverse-engineering the actual computation — the features the model represents and the circuits wiring them together.

That framing sets the bar. A useful interpretability result isn't "we understand the model." It's "we found the feature responsible for X, confirmed it causally, and changed X by intervening on it." Everything below builds toward that loop: find a feature → trace its circuit → verify by intervention.

The first thing that trips up newcomers: a single neuron is polysemantic — it fires for many unrelated concepts. That's because the model stores concepts as directions spread across many neurons, packing far more features than it has dimensions. This is superposition, and it works because features are sparse — few are active at once, so their interference rarely collides.

The full geometry — why d dimensions can hold many more than d near-orthogonal features, and why sparsity keeps them separable — is worked out in the Concept Circuits breakdown. The engineering consequence is what matters here: you cannot read features off individual neurons. You have to recover the directions first, which is exactly what the next tool does.

If features are superposed, how do you get them back? Train a sparse autoencoder (SAE) on a layer's activations: it re-expresses each activation vector as a sparse combination of a much larger dictionary of candidate features, with a sparsity penalty forcing only a few to be active at once. Done well, each learned feature becomes monosemantic — it means one thing you can name ("Golden Gate Bridge," "code comment," "a sense of danger").

The dictionary is over-complete (more features than neurons); the sparsity penalty is what makes each one crisp instead of a blur. This is the workhorse that turned interpretability from "stare at neurons" into "enumerate features."

Once you have a dictionary of named features, the rest of the toolkit becomes concrete: you can ask which features fire on an input, which drive an output, and what happens when you turn one up or down.

A circuit is the path of features the model uses to get from input to output. Attribution measures how much each feature contributes to a downstream one; chain those contributions and you have the circuit graph. In a linear toy — inputs → hidden features → output — the contribution of input i to output o is just the sum over hidden paths of the weights along the way:

And you can verify it causally: ablate input i (zero it) and the output must change by exactly attribution(o,i)·x[i]. Attribution that doesn't survive ablation is a story, not a circuit.

Real circuit tracing (activation patching, attribution graphs) is this idea scaled up through nonlinearities and many layers — but the discipline is identical: an attribution is only real if intervening on the cause changes the effect. The runnable tracer below builds the linear version end to end.

The payoff of naming a feature is that you can intervene on it. Clamp a feature's activation up and the model leans into that concept; clamp it down and the behaviour recedes. This is activation steering — the "Golden Gate Claude" demonstration is the famous example: amplify one feature and the model can't stop talking about a bridge.

For an engineer, steering is the difference between explanation and control. If you've found the feature behind an unwanted behaviour — a refusal misfire, a spurious correlation, a leaked style — you can often suppress it directly, without retraining. It's surgery on a specific direction rather than persuasion through prompts, and it's testable: the intervention either moves the behaviour the predicted way or your feature attribution was wrong.

Hold both halves honestly. Interpretability has gone from "stare at neurons and hope" to a genuine debugger with causal tools — but "we found the deception feature" is still far more aspiration than product at frontier scale. Treat it as a powerful partial debugger: it can explain and fix specific behaviours you go looking for; it cannot yet certify a model is safe. That gap is exactly why it's one of the most active frontiers in AI safety.