Doctors’ AI scribes get names of drugs and diagnoses wrong, NHS watchdog warns

Artificial intelligence tools being used to record NHS consultations have confused the names of medicines, incorrectly recorded diagnoses and omitted important treatment instructions, prompting warnings about the potential risk to patient safety.

Healthwatch England, the statutory patient champion, has raised concerns about errors made by AI-powered “scribes”, which listen to conversations between clinicians and patients before producing transcripts, clinical notes, summaries and letters. The technology is being increasingly adopted across the NHS as part of efforts to reduce administrative work and give doctors more time with patients.

Among the cases highlighted by Healthwatch was a woman whose AI-generated summary incorrectly stated that an MRI scan showed she had demyelination, a condition involving damage to the protective covering around nerves which can be associated with diseases including multiple sclerosis. The patient, who works within the NHS, challenged the record and it was subsequently corrected to “null demyelination”.

In another case, an AI scribe confused a medicine prescribed by a GP with a different drug with a similar name. Healthwatch also highlighted an occasion when an AI-generated hospital letter failed to record that a consultant had told a patient to obtain a repeat prescription for migraine medication from their GP. In each of the examples, the error was identified by the patient rather than the healthcare professional.

Healthwatch warned that inaccurate information could remain within patients’ medical records if errors are not identified and corrected, potentially affecting future treatment. It has called for greater clarity over how patients can report mistakes produced by AI scribing technology and how incorrect information can be removed or amended.

NHS guidance already makes clear that healthcare professionals remain responsible for the accuracy of information produced using AI scribes. Clinicians are required to check notes generated by the technology every time it is used and correct inaccuracies before information is added to a patient’s record. Patients should also be told when an ambient scribe is being used and can object to its use during their appointment.

Questions have also been raised about how the technology is regulated. The Medicines and Healthcare products Regulatory Agency clarified in July that AI scribing products used solely to transcribe or summarise clinical conversations, draft correspondence or suggest clinical codes for a clinician to review are not classed as medical devices under the current regulatory framework. However, systems intended to support diagnosis or treatment, or to take automated clinical actions without clinician review, are regulated as medical devices and must meet the relevant safety and performance requirements.

The warnings come as the NHS pushes ahead with wider adoption of ambient voice technology. In January, NHS England launched a national supplier registry and encouraged NHS organisations to consider the technology following research involving more than 17,000 patient encounters across nine NHS sites. The study found AI scribes increased direct interaction between clinicians and patients by 23.5% and reduced overall appointment length by 8.2%.

The potential benefits are significant. By automatically creating draft notes from consultations, AI scribes are intended to reduce the amount of time doctors spend typing or completing paperwork and allow them to concentrate more fully on the patient in front of them. NHS England has estimated the technology could save clinicians around two or three minutes per consultation.

However, concerns about accuracy illustrate the importance of maintaining human oversight as use of the technology expands. NHS England warns organisations to consider risks including information quality and “automation bias”, where people may be more inclined to accept an automated output without sufficiently questioning whether it is correct. Staff must understand that AI-generated material can contain inaccuracies and that responsibility for reviewing it remains with healthcare professionals.

Healthwatch’s intervention does not suggest AI scribes should be abandoned, but raises questions about whether safeguards and systems for correcting errors are keeping pace with their rapid adoption. With drug names, diagnoses and treatment instructions among the information being captured, even a seemingly small transcription mistake has the potential to become significant if it enters a medical record unnoticed.

As AI becomes a more routine presence in NHS consulting rooms, the challenge will be ensuring that technology designed to reduce clinicians’ workloads does not introduce new risks of its own. The benefits of allowing doctors to spend less time looking at screens and more time talking to patients are clear, but those benefits depend on AI-generated records being treated as drafts requiring careful human review rather than as automatically reliable accounts of what happened during a consultation.

Stay informed with the stories shaping health and social care.