A feature, in mechanistic interpretability, is a direction in activation space that stands for one human-meaningful concept. A sparse autoencoder (SAE) decomposes a layer’s dense, polysemantic activations into a large dictionary of sparse features, each firing for a specific idea — the building blocks an attribution graph then wires together.
Worked example: a sparse autoencoder is trained to reconstruct a model’s internal activations using a much larger but mostly-zero set of features, so each learned feature tends to correspond to one human-interpretable concept — a step toward reading what a network represents. Gotcha: it tackles superposition (models pack many concepts into few neurons) by spreading them into many sparse features, but the features are not guaranteed clean or complete, and interpreting them still needs human labeling — a research microscope, not a finished explanation of the model.