What happens when AI trains on AI? The risk of model collapse
‘The next phase of AI competition may be less about who has the biggest model and more about who has the cleanest data’
As AI-generated content spreads across the web, future models risk being trained on material produced by earlier ones. Researchers have recorded declining diversity, compounding errors and the loss of rare information under recursive training, although evidence of model collapse within frontier systems operating in production remains limited.
Kieran Gilmurray is an AI and digital transformation specialist with more than 30 years of experience. He is CEO of KG & Co, chief AI innovator at Technology Transformation Group and the author of books including The A-Z of Generative AI and Agentic AI and the Future of Work. He has also been recognised as a seven-time LinkedIn Top Voice.
In this exclusive interview with the Champions Speakers Agency, Kieran examines the evidence behind model collapse, the conditions under which synthetic data can improve performance and the controls needed to protect training datasets. He also explains why data provenance, expert verification and access to reliable human-generated information could shape the economics of the AI industry.
What measurable evidence suggests model collapse is beginning, and how far does that evidence extend to frontier models already in production?
There is some evidence that models can degrade when repeatedly trained on synthetic outputs from earlier models. Research shows this can lead to distribution drift, reduced diversity, rising error rates, and the loss of rare but important signals.
But I would be careful with the language here. I do not think we have clear public evidence that frontier models are already collapsing in production.
The near-term issue is more specific: the open web is becoming a lower-quality training substrate. That makes trusted data, provenance, expert feedback, and careful curation more valuable.
As AI-generated material spreads across the open web, how serious is the risk of it entering future training datasets unchecked?
Concerned, but not alarmed. The risk is not the synthetic data itself. Synthetic data can be extremely useful when it is carefully generated, properly validated, and combined with high-quality human or domain data.
The real risk is indiscriminate synthetic data: low-quality, unlabelled, recursively generated content pulled into future training sets without sufficient provenance or filtering.
The industry should treat this as a data supply chain problem. AI performance depends on the quality, diversity, and traceability of the knowledge it is trained on.
What does the research tell us about when synthetic training data degrades a model and when it improves performance?
The evidence supporting model collapse comes mainly from experimental and theoretical work. Studies show that recursive training on model generated outputs can narrow the distribution, amplify errors, and erase tail events. That matters because rare cases often carry risk, innovation, and differentiated expertise.
The evidence against a simplistic collapse narrative is that leading AI labs already use synthetic data successfully. In coding, mathematics, reasoning, and simulation, synthetic data can improve performance when it is filtered, tested, and evaluated.
So the right distinction is this: poor synthetic data creates degradation. High-quality synthetic data, governed properly, can be an asset.
What controls should AI companies build into their training pipelines to protect data quality and prevent recursive degradation?
- The most promising solutions are practical, not glamorous, but they matter.
- Better data provenance so companies know where training data came from.
- Stronger filtering to identify synthetic, duplicated, or low-quality content.
- More use of proprietary, domain specific, and expert-verified datasets.
- Human feedback loops that test outputs against real judgement.
- Synthetic data pipelines with validation, scoring, and rejection mechanisms.
- Right now, the industry needs more disciplined data engineering, not just larger models.
If reliable human-generated data becomes harder to source, how could that reshape competition, costs and barriers to entry across the AI industry?
This could become one of the defining competitive issues in AI. Most people assume the model is the moat. Increasingly, the data supply chain may be the moat.
If high-quality human-generated data becomes harder to source, the advantage shifts to companies with proprietary datasets, enterprise workflow data, expert evaluation loops, licensing relationships, and strong governance infrastructure.
It could also raise barriers to entry. Smaller model companies may find it harder to access enough trusted data, while larger players can pay for data and build feedback systems.
The next phase of AI competition may be less about who has the biggest model and more about who has the cleanest data, the strongest validation systems, and the deepest access to real-world expertise.
About Kieran Gilmurray
Kieran Gilmurray is a technology and AI expert specialising in digital innovation, intelligent automation and business transformation. With more than 30 years of experience, he is CEO of KG & Co and Chief AI Innovator at Technology Transformation Group.
A seven-time LinkedIn Top Voice, Kieran has written several books on emerging technology, including The A-Z of Generative AI, The A-Z of Organisational Digital Transformation and Agentic AI and the Future of Work. He advises organisations and speaks internationally on generative AI, data strategy, automation and the future of work.
This exclusive interview with Kieran Gilmurray was conducted by Tabish Ali of the Motivational Speakers Agency.