Data a model
writes for itself.

Data is the bottleneck for most AI work — real, labelled examples are scarce, expensive, and often missing exactly where you need them (rare cases, new formats, niche domains). Synthetic data sidesteps that: use a capable model to generate the examples. Ask a strong LLM to write instruction-response pairs, produce step-by-step reasoning traces, invent tricky test inputs, or paraphrase for augmentation — as many as you want, targeted at the exact distribution you're missing.

This powers a lot of modern practice: self-instruct (a model generating its own instruction-tuning data), distillation (a big teacher model labelling data for a small student), reasoning-trace generation (as in STaR), and eval-set creation. It's cheap, fast, and controllable. But generated data has a property real data doesn't: its quality is set by the generator, and generators are imperfect. That single fact drives everything else in this handbook.

The instinct with synthetic data is to generate more. The reality is that a pile of mediocre examples can make a model worse — training on wrong, repetitive, or off-distribution generations teaches wrong, repetitive, off-distribution behavior. What matters isn't how much you generate; it's how much survives a quality filter.

So the useful quantity is the effective dataset size: generate N examples, keep only the fraction p that pass a quality check, and you have p·N good examples. A filtered set of 300 strong examples routinely beats an unfiltered 1000. This is the lesson of the Phi models: trained on small, ruthlessly-filtered, "textbook-quality" data, they punched far above their parameter count. The filter — a test suite, a judge model, an execution check, a difficulty/diversity score — is not an afterthought; it's the most important part of the pipeline.

The subtler danger appears when synthetic data feeds back into the models that made it. Train a model on data generated by models — including its own output — recursively, without enough real data, and you get model collapse: each generation loses a little of the distribution's tails, the rare and unusual, and drifts toward the average.

Mathematically, each round of naive self-training multiplies the distribution's spread by a factor below one, so variance decays geometrically toward zero — the model forgets how to produce anything unusual. This isn't hypothetical: as more of the web becomes model-generated, models risk training on their own exhaust. The defenses are the same ones good pipelines already use — mix in real data to anchor the distribution, filter and deduplicate to preserve diversity, and never train purely on unfiltered recursive generations.

Two equations capture the whole tension. First, yield: generate N examples, keep the fraction p that pass the quality filter, and your effective, usable dataset is:

A small p can make a giant generation nearly worthless; improving the filter's pass-quality is usually worth more than generating more.

Second, collapse: naive self-training shrinks the distribution's variance by a factor s < 1 each round, so after k rounds the diversity has decayed geometrically:

Diversity vanishes exponentially under recursive self-training — which is why real data and filtering must stay in the loop. The runnable version below computes effective size, filters by quality, and watches variance collapse round by round.

A production synthetic-data pipeline is a generate-then-filter loop with real data in the mix:

The filter is where the value is — verify with the strongest cheap signal you have (unit tests for code, a judge model for text, execution for math), the same verify-then-keep logic behind STaR's reasoning bootstrapping. And curate for the target: if you need hard examples, filter for difficulty; if you need coverage, filter for diversity. Anchoring on real seeds and mixing real data back in are what keep the distribution honest and stave off collapse. Then evaluate on held-out real data — synthetic data can flatter itself, so the scoreboard must be real.

The quiet killer is distribution narrowing. Even one round of unfiltered self-generation subtly shrinks diversity — the model over-samples its favorite phrasings and under-samples the rare cases — and it compounds. So measure diversity, not just accuracy, and keep real data anchoring the set. A related trap: inheriting the generator's errors and biases. Synthetic data can only be as correct as its generator plus its filter; if the teacher is wrong in a systematic way and the filter can't catch it, the student learns the flaw at scale. Verify against ground truth wherever you can, not just against another model's opinion.

Two more. Evaluating on synthetic data is circular — a model can look great on data drawn from the same generator and fail on real inputs, so hold out real data for the scoreboard. And contamination: generated eval sets can accidentally overlap the training distribution, inflating scores. Used with discipline — strong generator, ruthless filter, real seeds and real evals — synthetic data is a superpower for covering the cases reality won't hand you. Used naively, it's a slow way to teach a model to be average.