SimCLR,
explained.

Labels are expensive; raw images are nearly free. Self-supervised learning tries to build good representations from unlabeled data by inventing a task the data itself can grade. SimCLR's invented task is contrastive: given a representation, judge whether two inputs are "the same thing seen differently" or "different things." No categories, no annotations — just same-or-different.

The elegant part is where the supervision comes from. Take one image, apply two random transformations — crop it here, jitter its colors, blur it — and you have two views that you know depict the same content. That known relationship is the label the model never had. Learn to pull those two views together in representation space while pushing away every other image, and you have learned what makes an image what it is, invariant to the incidental stuff.

Concretely: take a batch of images. For each one, create two augmented views. The two views of the same image are a positive pair — they should end up close. Every view of a different image in the batch is a negative — it should end up far. So each anchor view has exactly one positive to attract and many negatives to repel, all supplied by the batch itself for free.

Nothing here required a human to say "this is a dog." The batch is its own answer key: same image, positive; different image, negative. That is what makes it self-supervised, and why more images in a batch simply means more negatives to learn from.

The loss that enforces all this is NT-Xent — normalized, temperature-scaled cross-entropy. For an anchor view i with positive partner j, it is a softmax classification where the "correct class" is the positive and every negative is a wrong class. Similarities are cosine (that's the "normalized"), scaled by a temperature τ:

Minimizing this maximizes the positive's share of the softmax — it rises as the positive similarity grows, and falls as any negative creeps closer, so one loss both attracts the positive and repels negatives. The temperature τ controls how sharply the model focuses on the hardest (closest) negatives — smaller τ, sharper focus. And because the denominator sums over every negative, a bigger batch makes the task harder and the features stronger. The runnable version below shows the loss dropping as the positive dominates and rising as negatives multiply.

SimCLR's other contribution was measuring what actually matters. Three choices proved decisive:

The projection-head finding is a subtle, influential one: the representation you train the contrastive loss on is not the best representation to use downstream. Keeping the pre-projection features — which retain more information the loss would have thrown away — is now standard practice.

SimCLR closed much of the gap between self-supervised and supervised vision. Its headline results reset expectations for learning without labels:

The simplicity was itself a contribution: earlier contrastive methods leaned on extra machinery like memory banks. SimCLR showed that with enough negatives (big batch) and the right augmentations, you didn't need them — a cleaner recipe that others could build on.

SimCLR crystallized the contrastive family of self-supervised learning — learn by comparison, pull positives together and push negatives apart — and its influence runs straight into multimodal work: CLIP uses the same contrastive idea across images and text. The lineage of follow-ups (MoCo, BYOL, SimSiam) mostly explored how to keep the quality while dropping the need for huge batches of negatives.

It's illuminating to set it beside masked autoencoders, the other great self-supervised family. Contrastive methods need augmentations and negatives and learn invariances; masked methods need neither and learn by reconstruction. Two very different routes to the same destination — strong features without labels — and modern vision draws on both. SimCLR is the clearest statement of the contrastive route.