An implant could now help individuals who are unable to communicate by interpreting signals in their brain and converting them to understandable speech.
Researchers from the University of California—San Francisco, trained their model on five volunteers reading children’s books. As volunteers read a book, electrodes were used to measure brain voltages, and the patterns were then matched to speech.
Instead of using “mind reading,” the system reads the brain’s effort to control the lips, jaw, and larynx. Even those who don’t have the ability to speak can think thoughts to form their mouths to say certain words. The implant uses these signals to decipher what the brain is saying.
Although the implant is not perfected, currently, 69 percent of words can be understood using the system when given 25 options to pick from. By simulating the vocal tract, the researchers have been able increase the accuracy of the system.
“The relationship between the movements of the vocal tract and the speech sounds that are produced is a complicated one,” says lead researcher Gopala Anumanchipalli. “We reasoned that if these speech centers in the brain are encoding movements rather than sounds, we should try to do the same in decoding those signals.”
The research team came up with two neural networks, which included one network to match brain signals with movements of the vocal tract, and another one that turns the movement into synthesized speech.
“We still have a ways to go to perfectly mimic spoken language,” says another member of the team, Josh Chartier. “We’re quite good at synthesizing slower speech sounds like ‘sh’ and ‘z’ as well as maintaining the rhythms and intonations of speech and the speaker’s gender and identity, but some of the more abrupt sounds like ‘b’s and ‘p’s get a bit fuzzy.
“Still, the levels of accuracy we produced here would be an amazing improvement in real-time communication compared to what’s currently available.”
Although this may be the start of something extraordinary for individuals to use, it may be some time before this implant is ready for the market.
