DINOv2,
explained.

Self-supervised vision has three great families, each learning strong features without labels but by a different mechanism. Contrastive methods pull augmented views of one image together and push other images apart — they need negatives. Masked methods hide parts of an image and reconstruct them — they need a reconstruction target. Self-distillation — DINO's family — needs neither.

Its idea: don't compare against other images or reconstruct pixels. Just have one network (the student) learn to match the output of another (the teacher) on different views of the same image. The twist that makes it self-supervised is that the teacher isn't a separate, pretrained model — it's a slowly-updated copy of the student. There is no external signal at all; the model teaches itself. That should be a recipe for collapse, and the interesting engineering is why it isn't.

Both networks have the same architecture. The student is trained by gradient descent to make its output distribution match the teacher's on the same image (different augmentations — typically the student sees smaller local crops, the teacher global ones, so the student learns "local-to-global" correspondence). The teacher is not trained by gradients. Its weights are an exponential moving average (EMA) of the student's:

With a momentum m near 1, the teacher barely moves each step — it's a stable, lagged version of the student. As the student improves, the teacher inherits those improvements a beat later, and the student then chases the slightly-better teacher. The two bootstrap upward together. The runnable version below shows the EMA barely nudging the teacher per step, which is exactly what gives training its stability.

The obvious failure mode: student and teacher both learn to output the same constant for every image. That trivially minimizes the matching loss and learns nothing — collapse. DINO prevents it with two opposing operations applied to the teacher's output, which balance each other:

Centering alone would flatten everything toward uniform; sharpening alone would let one dimension run away. Applied together, they cancel each other's failure modes and leave a stable regime where the teacher's targets are neither degenerate constant nor uniform noise — exactly the informative signal the student needs. That delicate balance is the real trick that makes teacherless self-distillation work.

DINO (2021) established the method on ViTs and made a striking observation: its attention maps segmented objects for free, without ever being told what an object is. DINOv2 (2023) scaled the recipe into a foundation model for vision. Its contributions were less about the core loss and more about making it work at scale:

The result was a set of visual features you can pull off the shelf: freeze DINOv2, put a light head on top, and get strong results on classification, segmentation, depth estimation, and retrieval. It became a go-to image backbone precisely because those features generalized so broadly.

DINOv2's payoff is the generality of the features it learns, from images alone:

Where CLIP learns image features aligned to text, DINOv2 learns them from images alone — and the two are complementary. DINOv2's purely-visual features are especially strong on dense, geometric tasks where text supervision offers little, making it a common choice as a general vision backbone.

DINO and DINOv2 cemented self-distillation as a first-class self-supervised approach and gave vision a genuinely useful foundation backbone trained without labels. The finding that its features segment objects on their own hinted at something deep — that the right unsupervised objective recovers structure we usually think requires supervision. DINOv2 turned that hint into a practical, widely-used model.

Set beside its siblings, it completes a picture of self-supervised vision: contrastive (compare, with negatives), masked (reconstruct), and self-distillation (match a momentum teacher). Three different bets on what makes a good unsupervised objective, all reaching strong features — and DINOv2 is the clearest demonstration that the self-distillation bet, scaled with curated data, produces backbone-quality features. The through-line across all three: the label was never the only teacher.