Maybe you remember playing the game of telephone. You and your classmates would sit around in a circle and take turns whispering a secret message into your neighbor’s ear. By the time the message made it to the last person, you found out that what you heard wasn’t at all what others heard or said. Something like that seems to be happening to OpenAI’s Whisper application. It’s listening, but doesn’t always reliably reproduce what it heard. Let’s talk about what’s going on.

What is Whisper?

Whisper, created by OpenAI, uses AI to translate speech to text. In the context of healthcare, it is being used to create text versions of conversations between doctor and patient.

Why would doctors want to use Whisper?

Usually, doctors take what notes than can themselves or audio record their own impressions of a patient visit for later transcription. Capturing the entire conversation in the moment is a powerful amount data. No worrying about if you forgot something the patient said. Later on as the doctor is trying to determine a diagnosis or plan a treatment, they can always refer back to this complete set of notes about the patient. And of course, all this data together could be used to better make connections between patient experiences, disease, and outcomes.

That sounds great, so what has gone wrong with Whisper?

New reporting is out that Whisper isn’t always producing an accurate recitation of doctor/patient interactions. In fact, sometimes it just makes stuff up. When an AI model like this just makes something up, it is called hallucination. These hallucinations can be phrases, sentences, or more. They can misrepresent what was said or add in text that is completely out of context. For healthcare, this incorrect data could have serious implications. In the context of AI, it is unlikely anyone will read and correct all of the data transcriptions created. So if they use this flawed data as an input to a model, they will likely also make flawed conclusions about the output. Like many in the industry say, “garbage in, garbage out.”

If it can be unreliable, why would hospitals or clinics use Whisper?

When developing a new technology, there are often problems or errors encountered. This is all to be expected. The concerns comes with how much are we relying on this technology to actually influence patient care and is that a safe thing to do. To OpenAI’s credit, they have warned the industry that Whisper should be used with caution for certain applications. But with so many looking to do more, do it better, and do it faster, some warnings haven’t been heeded.

Learn More

What to know about an AI transcription tool that ‘hallucinates’ medical interactions

Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said

Introducing Whisper