Synthetic Data for Post-Training: When It Helps and When It Collapses

Synthetic data for post-training is doing two opposite things at once, and most discussions pick one and ignore the other. On one side, distilling reasoning traces from a strong teacher into a smaller student is among the most effective post-training moves of the last two years, the technique behind a whole generation of capable open models. On the other, training a model on data generated by recursively reusing its own outputs causes measurable degradation that ends in collapse. Same broad category, opposite outcomes. The difference is not whether the data is synthetic; it is whether the synthetic data adds signal the model did not already have, or recycles the distribution it already is. That distinction is the whole topic, and getting it right is what separates a cheap capability gain from a slow, hard-to-detect rot.

This is the eval-honesty beat applied upstream, to the training data itself. The evaluation crisis is about not trusting a number; this is about not trusting a corpus, and the failure mode is quieter because the damage hides in the tails of the distribution where your aggregate metrics never look.

Two regimes that share a name

The word "synthetic" covers two situations with different physics. Separating them is the first and most important move.

Distillation adds signal. When a stronger model (the teacher) generates solutions, reasoning traces, or labels, and a weaker model (the student) trains on them, the synthetic data carries information the student did not have: the teacher's competence, transferred. This is a one-directional flow from more capable to less, and it works precisely because the teacher's distribution is better than the student's. The open reasoning models that learned to think by training on traces distilled from frontier reasoners are the proof; this is the reasoning-model recipe in one sentence.

Self-generation recycles distribution. When a model trains on its own outputs, or on a chain of models each trained on the last one's outputs, no new signal enters the system. You are feeding a distribution back into the model that produced it, and the math of that loop is unforgiving. Shumailov et al., in their 2024 Nature paper, showed that recursive training on generated data leads to model collapse, and separated it into two phases: early collapse, where distributional errors accumulate and the model drifts off the true distribution, and late collapse, where low-frequency events permanently disappear and the output converges toward a low-variance point estimate that cannot be recovered.

The mechanism is why the two regimes diverge. Distillation is a transfer from a better distribution to a worse one, so it raises the student. Self-generation is a distribution sampling from itself, and every round of sampling loses a little of the tail, because rare events are by definition under-sampled. One adds information; the other can only lose it.

Worth naming the honest caveat on the canonical result, since eval honesty cuts both ways. Borji's follow-up note (arXiv 2410.12954) argues that what the Nature paper measured is a general property of iteratively refitting a distribution to samples drawn from itself, reproducible with kernel density estimators and no neural network in sight. That reading makes the result more portable, not less: it is not a quirk of transformer training you can engineer around with a better optimizer. But it also means the paper is a statement about a specific closed loop, not a prophecy about the open web.

How collapse actually shows up

Collapse is dangerous because of where it lands, not how big it is in aggregate. The center of the distribution holds up fine; perplexity on common text barely moves. The damage concentrates in the tails: rare facts, uncommon phrasings, minority dialects, edge-case reasoning, the long-tail knowledge that is exactly what makes a model useful past the obvious queries. An aggregate metric averages the healthy center with the rotting tail and reports something close to fine. This is the same trap as quantization, where perplexity hides the tail damage, and it has the same fix: you have to measure the distribution, not the mean.

The early signature is variance loss. A collapsing model becomes more confident, more repetitive, more uniform. Its outputs cluster; the diversity of its generations drops; it stops producing the unusual-but-correct answers it used to. By the time a benchmark score moves, the narrowing is well advanced. So the thing to watch is not accuracy, it is spread: the entropy of generations, the coverage of rare categories, the rate of distinct n-grams, the model's calibration on tail inputs. Falling diversity at steady accuracy is the leading indicator, and it shows up before the number you usually track does.

Research-grade diagnostics are starting to get sharper than surface counts. SIGMA (arXiv 2601.03385) tracks the eigenspectrum of the embedding Gram matrix and reports that under full recursion, where both data and weights carry forward each generation, geometric contraction of the representation space shows up well before surface-level degradation does. The honest caveat is scale: the experiments run on OPT-125M over a small technical corpus, so treat the spectral signal as a promising research probe rather than something validated at post-training scale. The cheap surface metrics (distinct-2, n-gram entropy, embedding-space spread) are what you can actually run on a nightly job today.

The rule that prevents it

The cleanest empirical result in this literature is also the most actionable. Gerstgrasser et al. (arXiv 2404.01413) showed that collapse in the Shumailov setup depends on a specific assumption: that each generation replaces the previous data. When you instead accumulate, adding synthetic data on top of the real data rather than substituting for it, collapse is largely prevented. They back it both ways: empirically across models and modalities, and analytically in a linear-model framework where accumulating data gives test error a finite upper bound independent of the number of iterations, while replacing it lets the error grow without bound.

The follow-up (Kazdan et al., "Collapse or Thrive?", arXiv 2410.16713) sharpens the boundary in a way that matters operationally. Accumulate-and-train-on-everything stays stable even as the real-data proportion trends toward zero, because the absolute quantity of real data never shrinks. But a third workflow, where data accumulates while each generation trains on a fixed-size random subset, sits between the two: the real anchor gets diluted out of the sampled mixture even though it still exists on disk. The lesson is not "keep some real data somewhere." It is that the anchor has to be present in the batches the model actually sees.

That single design choice, accumulate rather than replace, is most of the practical defense. Treat synthetic data as an additive supplement that fills specific gaps (more examples of an under-represented skill, more coverage of a format), not as a replacement that lets you retire the expensive real corpus. The moment synthetic data becomes the majority of what your sampler draws and the real anchor thins out of the mixture, you are back in the replacement regime, and the tails start to go.

What 2026 post-training actually does

The theory above describes a closed recursive loop. Almost nobody in production is running one, and the gap between the collapse literature's setup and current practice is worth stating plainly, because it is where most of the confusion lives.

Look at what recent open post-training pipelines document. NVIDIA's Nemotron 3 line reports synthetic data generated by other labs' models as teachers: Qwen3.5-397B-A17B, Qwen3.5-122B-A10B, and gpt-oss-120b among them, plus domain-specific synthetic sets and synthetic rephrasing of Common Crawl. Nemotron-Cascade 2 (arXiv 2603.19220) builds a 30B MoE with 3B active on roughly 4.4M math and 4.2M code SFT samples drawn from responses by DeepSeek-V3.2, gpt-oss-120b, and Qwen models, then runs multi-domain on-policy distillation from domain-specific teacher checkpoints during RL.

Two things follow. First, this is heterogeneous cross-model distillation, not self-generation: signal enters from outside the model being trained, which is the safe regime, and the ecosystem's real risk is a slow homogenization across labs (everyone distilling from the same handful of teachers) rather than a single model eating itself. Second, note whose numbers those are. Nemotron-Cascade 2's headline results, including IMO 2025 gold-level scoring and 92.4 on AIME 2025 with tools, are the authors' own reported figures on their own pipeline, and the comparisons against Qwen3.5-35B-A3B are theirs to frame. That is normal, and it is still vendor-measured. The relevant question for your team is not whether their number is real but whether their recipe transfers, and a synthetic-data recipe is exactly the kind of thing that does not transfer cleanly, because the teacher, the filter, and the eval are all doing work the paper compresses into one line.

Where synthetic data genuinely earns its place

The pessimism has limits, because synthetic data is not just safe-when-anchored, it is sometimes the best tool available. The productive uses share a property: they introduce signal from outside the model's own distribution, or they filter rather than generate.

Distillation from a stronger teacher is the headline case: real signal flowing downhill. It is why a small student can learn reasoning behavior far above what its size and its own pretraining would predict, and it is a large part of why open models train as cheaply as they do.

Verifiable synthetic data is the safest kind, because a verifier supplies external truth. Generating math problems and checking the answers, generating code and running the tests, generating data and filtering it through a deterministic checker: the filter is the signal source, and it is hard to game in the way the RLVR loop is hard to game. You are not trusting the model's outputs; you are keeping only the ones an external check certifies. This sidesteps the recursive-recycling problem, because the verifier, not the model's own distribution, decides what survives.

Targeted gap-filling uses synthetic data to cover a known deficiency (a rare intent, an under-represented language, a format the real data lacks) as a supplement to a real-data base. Bounded, additive, and anchored, this is the everyday productive use, and it is firmly in the accumulate regime.

What ties the safe uses together is that the signal comes from somewhere other than the model recycling itself: a stronger teacher, an external verifier, or a real-data anchor that holds the distribution open. When synthetic data has one of those, it is a cheap capability gain. When it has none of them, it is a slow loss disguised as free data.

The loop is bounded, even when it works

There is a third position between "distillation is free capability" and "self-generation collapses," and it is where most self-improvement proposals actually land: loops that work, and then stop working, faster than the pitch implies. This blog's standing position on feedback loops is that every observed one shows bounded improvement with diminishing returns, and the synthetic-data literature keeps re-deriving that result from a new direction.

Two 2026 preprints make the boundary concrete. "Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?" (arXiv 2603.24472) reports that self-distillation on solution-guided data can improve coding and chemistry while degrading competition math, with SFT on solution-guided traces dropping DeepSeek-R1-Distill-Qwen-7B's AIME24 score sharply against its own base. Their proposed mechanism is worth the attention: conditioning the teacher on the full solution suppresses uncertainty markers ("wait," "hmm," "perhaps"), the student imitates that confident, compressed style, and the style fails on problems it has not effectively seen. The synthetic data was correct and the distillation still cost you something, because what transferred was a rhetorical posture, not just an answer.

The other boundary is the verifier. "When Good Verifiers Go Bad" (arXiv 2606.14629) runs verifier-driven self-improvement loops on a small VLM student and finds two things: on one benchmark every verifier in the ladder pushed the student below its frozen baseline while training loss went down, and above a competence threshold, extra verifier quality bought very little (roughly a 20-point gain in verifier rubric accuracy yielding no more than about 4 points of student improvement). Both are preprints on a small student model, so hold them as a shape rather than a constant. But the shape matches the RLVR position exactly: the verifier is the ceiling, a bad verifier is worse than none, and a better verifier stops paying long before you stop spending on it. Self-generation does not have to reach full collapse to be a bad trade. It only has to stop returning more than it costs, and it does that early.

A protocol you can run this quarter

None of the above is actionable until it is a procedure. Run this on any post-training mixture that contains synthetic data, before the run and on a schedule after it.

1. Classify every synthetic source by where its signal comes from. For each generator in the mixture, write down one of: stronger external teacher, external verifier/filter, or the model being trained (or a sibling of it). Sources in the third bucket are the ones under suspicion. If you cannot name the signal source for a slice of data, that slice is the third bucket by default.
2. Measure the real anchor as a fraction of sampled batches, not of storage. The Kazdan et al. result is that dilution happens in the sampler. Log the real-data share your dataloader actually draws each epoch, and alert when it falls below the floor you set (pick a floor, write it down, defend it).
3. Freeze a diversity baseline before you train. On a fixed prompt bank covering both common and deliberately rare inputs, record generation entropy, distinct-2, self-BLEU or an embedding-space spread measure, and the rate of refusal-to-commit tokens. This is the pre-image you will diff against. Doing this after you notice a problem is too late, because you will have nothing to compare to.
4. Build a tail eval, not just a benchmark. Aggregate scores are the metric that hides collapse. Carve out slices for rare entities, low-resource languages or dialects you serve, long-horizon reasoning, and inputs your production logs show fewer than N times. Report per-slice, never averaged.
5. Run the diff after every synthetic-data round. The trigger condition is falling diversity at flat-or-rising accuracy. That combination, not a benchmark drop, is what says stop. A benchmark drop means you are already late.
6. Validate the verifier before you trust the loop. If synthetic data is filtered or scored by a model, measure that model's agreement with ground truth on a labelled holdout first. A verifier below its task baseline actively steers training the wrong way, so this check gates the whole pipeline.
7. Budget the loop as bounded from the start. Plan two or three rounds, define the stopping criterion before round one (marginal gain per round below X on the tail eval), and re-verify. Compounding self-improvement is not the observed behavior; plateaus are.
8. Keep a rollback anchor. Keep the pre-synthetic checkpoint and its diversity baseline. Collapse is described in the literature as not recoverable by continued training on the collapsed distribution, so the cheap fix is having a checkpoint to go back to, not a clever repair.

The honest framing for builders

Synthetic data is neither the free lunch the optimistic framing sells nor the poison the collapse headlines imply. It is a tool with a sharp boundary, and the boundary is the direction of information flow. Flowing in from a better source (a teacher, a verifier, real data), it helps, often a lot. Flowing in a circle from the model back to itself, it degrades, quietly, from the tails inward, on a timeline that beats your usual metrics to the punch. And even in the good regime, it is a lever with a stop, not an engine that compounds.

So the discipline is: prefer distilled and verified synthetic data over self-generated; accumulate, never replace, and check that the anchor survives your sampler; validate the verifier before you trust anything it filters; monitor distributional diversity, not just accuracy; and budget every self-improvement loop as bounded from the first round. Do that and synthetic data is one of the highest-leverage inputs in post-training. Skip it and the corpus rots from the inside while the dashboard says everything is fine.

Key Takeaways

1. "Synthetic data" covers two opposite regimes. Distillation from a stronger teacher adds signal and raises the student; self-generation recycles the model's own distribution and can only lose information. The direction of information flow, not the synthetic label, decides the outcome.
2. Self-recycling collapses from the tails inward. Recursive training on generated data drifts off-distribution first (early collapse) and then permanently erases low-frequency events (late collapse), converging toward a low-variance center (Shumailov et al., Nature 2024). Borji's follow-up note argues this is a general property of refitting a distribution to its own samples, which makes it more portable, not less.
3. Collapse hides in aggregate metrics. The center holds and perplexity barely moves while rare facts, edge cases, and minority distributions rot. Measure the distribution, not the mean, the same trap as quantization's hidden tail damage.
4. Falling diversity is the leading indicator. A collapsing model gets more confident, repetitive, and uniform before any benchmark score moves. Track generation entropy, rare-category coverage, and distinct-n-gram rate against a frozen pre-training baseline, not just accuracy.
5. Accumulate, do not replace, and check the sampler. Gerstgrasser et al. show accumulation bounds the test error where replacement lets it grow without bound. Kazdan et al. add the operational catch: training each generation on a fixed-size subset dilutes the real anchor out of the batches even while it still sits on disk.
6. Verifiable synthetic data is the safest kind, and only as good as the verifier. An external checker (run the tests, verify the math) is the signal source that sidesteps recycling. But a verifier below its task baseline steers training the wrong way, and above threshold extra verifier quality pays very little, so validate it against ground truth before you trust the loop.
7. Production distillation is cross-model, and the numbers are vendor-reported. Open 2026 pipelines (Nemotron 3, Nemotron-Cascade 2) distil from other labs' models, which is the safe regime; the ecosystem risk is homogenization across a few shared teachers, not one model eating itself. Their headline scores are self-reported on their own pipelines, and a synthetic-data recipe is exactly the kind of thing that does not transfer cleanly.
8. Even the loops that work are bounded. Self-distillation can improve one domain while degrading another (2026 work reports competition-math regressions from correct, solution-guided traces), and verifier-driven self-improvement plateaus fast. Budget two or three rounds with a stopping criterion written down in advance, and keep the pre-synthetic checkpoint.

The Acing AI newsletter covers data the way this piece does: where the corpus rots, not just where it grows. Subscribe for the version that watches the tails.