Model Collapse Is Happening, We Just Pretend It Isn't (opens in new tab)

It isn’t pointless.

The author cited research that demonstrates that model collapse can happen on a small scale.

The author also cited sources that a larger and larger portion of the web will be written by language models.

There are already studies showing that LLM generated text is less diverse than human generated text:

https://techxplore.com/news/2026-03-llms-creativity-ai-respo...

https://arxiv.org/html/2501.19361

The studies don’t show that the lack of creativity in LLMs is caused by model collapse or that the problem is getting worse.

But 1) we know they do this and 2) we know that training on synthetic data can cause model collapse.

> It's pointless to write a whole article about how model collapse is actually happening and isn't just a theoretical concern with no evidence that model collapse is actually happening.

Except perhaps the link to article on the peer-reviewed paper that describes the problem in detail.

https://www.cs.ox.ac.uk/news/2356-full.html

> Researchers at Oxford and Cambridge published work on this back in 2023, showing how iterative training on synthetic data leads to progressive degradation.

The fact that you aren't persuaded is the evidence of collapse. Previous generations of LLMs persuaded everyone of anything.

Oh, so this isn't about the Modell's collapse? https://www.nytimes.com/2020/03/11/business/modells-bankrupt...