HackerHACKER PAPER · 2024 · arXiv preprint
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
What question does this paper answer?
Must training each generation on the previous model’s output inevitably make models worse? This paper separates replacing old data with newly generated data from retaining the initial dataset and accumulating outputs across generations. It compares these processes in language, molecular-conformation, and face-generation experiments, then analyzes error in a particular linear-regression framework. Accumulation substantially changes degradation, but avoiding collapse does not mean that every quality measure remains unchanged forever.
What it contributes
- 01
Retaining the initial dataset and outputs from earlier generations is a different training regime from keeping only the latest model’s outputs. The language-model experiments observe different test-error trends between these regimes.
- 02
Under explicit distributional, noise, sample-size, and underparameterization assumptions for linear regression, the authors derive a test-error bound independent of the iteration count. This is not a theorem about arbitrary large language models.
- 03
The VAE with accumulated face data still shows increased test error and fewer generated details. The paper distinguishes unbounded error growth from other meanings of collapse, including loss of modes.
Check the original passages
Pages count from the first page of the PDF file. Page jumps depend on the source viewer; section names and short quotes provide another way to find the passage.
avoiding model
collapse refers instead to bounded error over such iterations.the test error of accumulating data does increaseGuide source: arXiv v2 ↗
Read this paper alongside other sources
These readings connect the paper with specific arguments from talks and other sources.
Hacker News comments 6
Verified mentions linking to this paper, newest first. This is not the complete comment tree of any one submission. Recommendations, references, and criticism are labeled separately; comments remain in their original language.
Which studies? [edit: I'll assume you mean these two given by @dorolow: https://arxiv.org/abs/2404.01413 https://arxiv.org/abs/2406.07515]
> It can't be used as the -only- data source of course, but that's not how it is being used
Right, so human data creation would also have to scale up exponentially, and that's not gonna happen.
> because there's no difference between synthetic data and the data people can create
I mean, that's obviously false, otherwise model collapse wouldn't exist. The difference is statistical, but it's there.
> It is accelerating how quickly researchers and engineers can do their jobs. > https://news.mit.edu/2026/ai-helps-design-new-materials-that...
That's pretty clearly a hype article, the headline even says "The CrysVCD tool developed at MIT COULD cut the huge amounts of time and money spent". I'm asking for empirical measurements of timelines, not hypotheticals.
> This is only the beginning, too... Look ahead a year or two.
Lol that excuse is getting really old
Edit: https://arxiv.org/abs/2404.01413 https://arxiv.org/abs/2406.07515
It used an incorrect theoretical framing that assumed that data was being replaced rather than accumulated as a result of more training (see https://arxiv.org/abs/2404.01413 which explores this). This is incorrect because this simply isn't how real-world datasets are created via synthetic data generation (which generally accumulate more data over time rather than replace their data). As a result most of the theoretical results were invalid.
Empirical evidence has also cast a considerable amount of doubt on the paper. For example, Microsoft Phi-4 was an empirical test in specifically what happens if the majority of your training data is synthetic rather than human and it turns out that Phi-4 did significantly better than previous models which relied primarily on human data.
There's some nuance to all of this in how exactly you do this, but the original claims of the paper are looking really shaky at this point.
The original model collapse paper assumes you train networks on 100% synthetic data produced by the previous generation. But if you maintain some portion of real data then the problem is mitigated.
Read 2 more comments
https://news.ycombinator.com/item?id=36319076
Some research since seems to add nuance to its conclusions:
https://arxiv.org/abs/2404.01413
Model collapse itself is(was?) a fairly serious research topic: https://arxiv.org/abs/2305.17493
We've by now reached a "probably not inevitable" - https://arxiv.org/abs/2404.01413 argues there's a finite upper bound to error - but I'd also point out that that paper assumes training data cardinality increases with the number of training generations and is strictly accumulative.
To a first order, that means you better have a pre-2022 dataset to get started, and have archived it well.
but it's probably fair to say current SOTA is still more or less "it's neither impossible nor inevitable".
Direct submissions and discussions (4)
Avoiding Model Collapse via Accumulating Real and Synthetic Data · 5 points · 2024-07-24 ↗Is Model Collapse Inevitable? Breaking the Curse of Recursion · 2 points · 2024-06-02 ↗Is Model Collapse Inevitable? · 5 points · 2024-05-02 ↗Is Model Collapse Inevitable? Breaking the Curse with Real and Synthetic Data · 3 points · 2024-04-08 ↗