The Scribe That Heard Everything and Understood Too Little

Imagine hiring the fastest note-taker you have ever met. They catch almost every word of every consultation. Then, mid-medication review, the doctor says, “Do not restart the metformin.” Your new hire writes, “We’ll restart the metformin.”

The sentence reads perfectly. Nobody notices.

This is not hypothetical. It happened when we evaluated a demo version of ElevenLabs Scribe v2 Medical. Its maker calls the model family “the most accurate speech-to-text model.”

Why the Stakes Are High

AI scribes are gaining popularity because they help doctors save time. One study saw clinician burnout fall from 51.9% to 38.8% after 30 days of use. But every scribe starts by turning speech into text, and any error there carries into the patient's chart. Even older speech-recognition drafts had a 7.4% error rate in clinical documents. That dropped to 0.3% only after human review.

What the Test Found

The team ran nine realistic doctor-patient conversations. They included patients who correct themselves, relatives who interrupt, and doctors who change a dose mid-sentence. The specialties ranged from cardiology to pediatrics. Each transcript was compared line by line with what was actually said.

• Guideline adherence averaged 27.92%. The best conversation reached 35.7%. Seven of nine landed between 28.2% and 35.7%, which suggests a ceiling rather than a few bad days.
• 5 of 9 had a wrong drug name or dose. Methotrexate became tofacitinib, and losartan became rosuvastatin.
• 5 of 9 had a wrong number or lab value.
• 5 of 9 changed a doctor's decision or reasoning.
• 0 of 9 had speaker labels. Doctor, patient, and relatives ran together in one block.
• 2 of 9 transcribed background stock-market news as part of the consultation.

To be fair, it did well on some visits. In one busy pediatric consult, every number, dose, and drug was right, and it even kept the doctor's correction from 5 ml to 8 ml.

That is the real risk. The errors sit inside transcripts that otherwise look trustworthy.

The Takeaway

We often judge speech AI by how many words it gets right. Medicine does not work that way. A transcript can be 95% correct and still be dangerous if the 5% it misses is the drug, the dose, or the word “not.”

A few practical lessons follow:

• Clinicians: Treat the transcript as a draft, not a record. Check every drug, dose, and lab value, and read the plan twice.
• Scribe Builders: Turn on speaker labels and key term prompting. Add automatic plausibility checks for drugs and numbers.
• Health Systems: Test on your own messy audio before you buy. Re-test the exact version you deploy, because the report tested a demo version, and production settings may differ.
• Patients: Ask your clinician to review what goes into your record. A drug you never took, or a family history you shared, could be wrong or missing.

Fluent is not the same as faithful. Until these errors are caught automatically, a clinician's review should stand between the transcript and the chart.