GPs force NHS England to hit pause on a ‘ground-breaking’ AI project

GPs have asked NHS England to refer itself to ICO to ensure correct process

GP leaders have halted progress on a predictive AI model based citing concern about whether the correct procedures were followed. The pause is likely to frustrate those advocating for greater use of AI within healthcare.

NHS England has paused a project using GP data to train an AI model, following concerns raised by GP professional associations the BMA and RCGP. Both groups said they were unaware that GP data was being used by NHS England to train ‘Foresight’ – a GPT based LLM.

Foresight is a LLM specifically designed for healthcare and has been trained in the NHS England Secure Data Environment (SDE), a secure data and research analysis platform, on anonymised NHS data from millions of people in England.

The model is designed to recognise patterns in the structured and unstructured data contained within electronic health records and make predictions accordingly. It was performing well in tests by clinicians assessing the accuracy of predictions generated.

However, the BMA and RCGP have temporarily halted the project, stating that said they were not aware that GP data, collected for Covid-19 research, was being used to train the model.

Researchers working on the model emphasised the routine and anonymised nature of the training data.

However, GP leaders said that it was unclear whether the correct processes were followed to ensure that data was shared in line with patients’ expectations and governance processes.

The BMA and RCGP joint IT committee has asked NHS England to refer itself to the ICO over this issue ‘so the full circumstances can be understood’. The committee also demanded that NHS England pause ongoing processing of data as a precaution.

BMA GP committee England deputy chair Dr David Wrigley said: “For GPs, our focus is always on maintaining our patients’ trust in how their confidential data is handled.

“We were not aware that GP data, collected for Covid-19 research, was being used to train an AI model, Foresight.

“As such, we are unclear as to whether the correct processes were followed to ensure that data was shared in line with patients’ expectations and established governance processes.

“We have raised our concerns with NHS England through the joint GP IT committee and appreciate their verbal commitment to improve on these processes going forward.

“The committee has asked NHS England to refer itself to the Information Commissioner so the full circumstances can be understood, and to pause ongoing processing of data in this model, as a precaution, while the facts can be established.”

RCGP chair Professor Kamila Hawthorne said: “As data controllers, GPs take the management of their patients’ medical data very seriously, and we want to be sure data isn’t being used beyond its scope, in this case to train an AI programme.

“Patients need to be able to trust their personal medical data is not being used beyond what they’ve given permission for, and that GPs and the NHS will protect their right to data privacy. If we can’t foster this patient trust, then any advancements made in AI – which has potential to benefit patient care and alleviate GP workload – will be undermined.”

The announcements by the BMA and RCGP come days after Sir Tony Blair’s pronouncement that Britain needs to stop being “squeamish” about AI and embrace a future of AI informed clinicians.

Speaking at the SXSW London festival, the former Prime Minister said that fears about artificial intelligence should be outweighed by the “absolutely transformative” impact it could have on public services like healthcare and education by saving time and money.

“When I stand back and I look at what [AI] is doing, I think we’re in the foothills of the most transformative revolution since the Industrial Revolution of the 19th century,” he told the festival audience.

The country cannot afford to be squeamish about the power of this emerging technology, said Blair, adding: “It’s absurd we haven’t yet put all of our NHS data there available to be used for innovation.”

Last year, a major Government-commissioned review found that establishing a central system allowing access to GP data for research was England’s ‘highest data priority’.

Flann Horgan, Head of Healthcare Sector, UK & Ireland at NTT DATA brings it back to the data.

He said: "The real barrier to Sir Tony Blair’s vision of AI doctors is secure access to high-quality data. For over 30 years, we’ve seen localised attempts at digitisation, from patient portals and shared care records to regional pilots. But the legacy systems on which they sit weren’t built to support truly integrated AI care.

"I’ve seen first-hand how AI can support clinicians instead of replacing them. The Royal Marsden is already developing and evaluating AI algorithms to improve the accuracy of cancer diagnosis. But that impact depends entirely on having fast, accurate data available at the point of care. Without interoperability and clear data standards, AI risks becoming just another layer of complexity, another system to juggle, rather than a tool that genuinely improves care.

“We have a golden opportunity to reframe the patient experience, but data is not a static resource. Its value depends on ensuring clinicians can trust and act on AI insights. And the value of AI itself lies in its ability to enhance clinical judgement, not replace the doctors who drive patient care forward every day.”