Blend the
examples.

Standard training minimizes loss at the exact training examples — and nowhere else. That's called empirical risk minimization, and it leaves the model free to do whatever it likes between the data points, which is most of input space. Neural networks tend to fill that void with jagged, wildly confident decision boundaries: they'll assign 99.9% confidence to regions they've never really seen, memorize noisy labels, and flip their answer under tiny adversarial perturbations. The trouble is precisely the space in between.

mixup (Zhang, Cisse, Dauphin & Lopez-Paz, 2018) addresses this with a startlingly simple idea: train on convex combinations of pairs of examples. Take two training points, blend their inputs with some weight λ, blend their labels with the same weight, and train on the result. This is vicinal risk minimization — you're minimizing loss not just at the data points but in the interpolated neighborhood between them, teaching the model to transition linearly from one class to another rather than snapping discontinuously. Two lines of code, no new parameters, and a measurably better model.

The crucial detail is that mixup blends the labels too, not just the inputs. If you averaged two images but kept a single hard label, you'd be lying to the model about a half-and-half picture. Instead the label becomes a soft target: a 70/30 image gets the target "0.7 of class A, 0.3 of class B." The blend is honest, and the loss (cross-entropy against the soft target) is meaningful.

At λ = 1 you recover example i exactly; at λ = 0, example j; in between, a genuine mixture with a matching soft label. Because the same λ weights both input and label, the mapping stays consistent — the model is told "this input is this fraction of the way from A to B, and so is its answer." Repeated over many random pairs, it learns to interpolate smoothly across the whole data manifold.

How aggressively should you mix? That's set by drawing λ from a Beta(α, α) distribution — a symmetric distribution on [0, 1] with a single hyperparameter α that shapes it:

Because Beta(α, α) is symmetric, its mean is always 0.5, but α decides whether the mass piles up at the extremes (gentle) or spreads toward the middle (aggressive). Small α keeps most blends close to a real example with a light dusting of a second; that's usually where the gains live. Push α too high and you drown every example in a 50/50 mush, which can underfit. It's a single dial trading off "stay near real data" against "explore the space between," and modest values tend to win.

The entire method is two equations sharing one mixing weight:

(x_i, y_i) and (x_j, y_j) are two examples with one-hot labels y. x̃ is the blended input; ỹ is the blended label (a valid probability distribution, since a convex combination of one-hot vectors sums to 1). Train on (x̃, ỹ) with the usual cross-entropy loss.

Two properties make it well-behaved. First, it's a genuine convex combination: the weights λ and 1−λ are non-negative and sum to 1, so x̃ lies on the segment between the two inputs and ỹ stays a proper probability distribution. Second, it degrades gracefully — at the endpoints it is ordinary training on a single example, so mixup only ever adds interpolated examples to the mix. The runnable version below implements the input blend, the label blend, and checks these endpoint and simplex properties.

For two lines of code, the payoffs were broad:

It isn't a universal win: on some tasks the blends are semantically odd (mixing two very different modalities, or data where linear interpolation isn't meaningful), and very high α can underfit. The gains are usually modest per run rather than dramatic. But the cost is so close to zero — a couple of lines, no extra parameters, negligible compute — that the trade almost always favors turning it on.

mixup became a standard ingredient of the modern training recipe, especially in vision. It's a default in many strong image-classification pipelines and part of the "bag of tricks" that squeezes extra accuracy out of ResNets and Vision Transformers — indeed, the heavy augmentation regimes that make data-hungry ViTs trainable lean on mixup and its relatives heavily. Its close cousin CutMix pastes a patch of one image into another and mixes labels by area; the two are frequently used together.

Conceptually, mixup elevated data augmentation from a bag of ad-hoc image tweaks (crop, flip, rotate) to a principled tool grounded in vicinal risk minimization — reshaping what counts as a training example rather than just perturbing pixels. It sits alongside other cheap, architecture-free regularizers like dropout in the standard toolkit, and its calibration and robustness benefits made it valuable wherever trustworthy confidence matters, not just raw accuracy. That so trivial an intervention — average two examples, average their labels — could reliably improve generalization was itself the lesson: sometimes the biggest lever is on the data, not the model.