One Brain Implant Decoded Speech and Gestures at the Same Time

A brain implant has let people with severe paralysis attempt words and upper-body gestures at the same time, then watch both appear through a digital avatar. The system is the first brain-computer interface reported to decode these two parts of communication in parallel from one implant.

The open-access study, published in Nature Neuroscience on September 14, used electrocorticography, or ECoG. A grid of 253 electrodes placed on the surface of the sensorimotor cortex recorded electrical activity associated with attempted speech and movement. Separate neural networks then classified phrases and gestures while running together.

Three people with paralysis contributed data. Two used the complete avatar system for combined communication. One participant, who had a brainstem stroke, silently attempted five phrases and four gestures. Another participant with amyotrophic lateral sclerosis vocalized ten phrases and imagined ten gestures, including a wave, shrug and thumbs-up.

In the main real-time simultaneous test, the system identified speech with 70% accuracy and gestures with 66% accuracy across ten choices of each, compared with a chance level of 9.1%.

During a smaller conversational demonstration with the participant who had ALS, average accuracy reached 75% for speech and 85% for gestures across five blocks. Decoded words appeared as text while the avatar moved. The result extends earlier work that restored one channel at a time. NewTqnia previously covered a different brain implant used independently at home for more than 3,800 hours.

Why simultaneous training mattered

Speech and body movements recruit partly overlapping regions of the motor cortex. Models trained only on isolated words or isolated gestures performed worse when both were attempted together. Adding combined examples to training reduced missed signals and false activations, showing that a decoder cannot simply assume two simultaneous intentions equal the sum of two separate ones.

The US National Institutes of Health, which funded the work, describes it as a step toward fuller digital expression. An independent Nature assessment likewise calls the device a proof of concept rather than a clinical product.