New brain tech turns paralyzed patient’s thoughts into speech with 97% accuracy

A new brain-computer interface (BCI) developed at UC Davis Health can translate brain signals into speech with an impressive accuracy of up to 97%—claimed to be the highest accuracy ever achieved by such a system.

The researchers implanted sensors in the brain of a man with severe speech impairment caused by amyotrophic lateral sclerosis (ALS). Remarkably, he was able to communicate his thoughts within minutes of using the system.

“Our BCI technology helped a man with paralysis to communicate with friends, families and caregivers,” said David Brandman, UC Davis neurosurgeon and co-senior author of the study. “Our paper demonstrates the most accurate speech neuroprosthesis (device) ever reported.”

What is ALS?

Amyotrophic lateral sclerosis (ALS), often referred to as Lou Gehrig’s disease, disrupts the functioning of nerve cells responsible for controlling bodily movements.

As the condition progresses, individuals gradually lose their ability to stand, walk, and use their hands. Additionally, it can impair the muscles required for speech, eventually leading to a complete loss of clear communication.

A novel technology is being created to help individuals who are unable to speak due to paralysis or neurological disorders like ALS. This innovation deciphers brain signals when a person attempts to speak, converting them into text, which the computer then vocalizes.

Functioning of the UC Davis BCI

When a user tries to articulate words, this advanced brain-computer interface (BCI) device captures the corresponding brain signals and converts them into text displayed on a screen. The computer subsequently reads the text aloud.

To refine this system, researchers enrolled Casey Harrell, a 45-year-old ALS patient, in their clinical trial. At the time of participation, Harrell exhibited significant weakness in both arms and legs (tetraparesis), and his speech was so impaired (dysarthria) that he required assistance for communication.

In July 2023, Dr. Brandman performed the implantation of the experimental BCI device. He inserted four microelectrode arrays into the left precentral gyrus, the brain region associated with speech coordination. These arrays are designed to capture brain activity using 256 cortical electrodes.

“We’re really detecting their attempt to move their muscles and talk,” explained neuroscientist Sergey Stavisky, co-principal investigator of the study.

“We are recording from the part of the brain that’s trying to send these commands to the muscles. And we are basically listening into that, and we’re translating those patterns of brain activity into a phoneme — like a syllable or the unit of speech — and then the words they’re trying to say.”

‘Transformative’: 97% accurate speech interpretation

Harrell utilized the system in both guided and free-flowing conversations. In each scenario, the device decoded his speech in real-time, with the system continuously updating to maintain its precision.

The decoded words were displayed on a screen and impressively spoken aloud in a voice resembling Harrell’s own, as it sounded before his ALS diagnosis. This voice was synthesized using software that had been trained on audio recordings of his voice from before the onset of the disease.

During the initial training session for speech data, the system achieved an impressive 99.6% accuracy with a vocabulary of 50 words within just 30 minutes.

“The first time we tried the system, he cried with joy as the words he was trying to say correctly appeared on-screen. We all did,” Stavisky said in the press release .

“At this point, we can decode what Casey is trying to say correctly about 97% of the time, which is better than many commercially available smartphone applications that try to interpret a person’s voice,” Brandman said.

“This technology is transformative because it provides hope for people who want to speak but can’t. I hope that technology like this speech BCI will help future patients speak with their family and friends.”

A study about this work was published in the New England Journal of Medicine .