Aleph Neuro Achieves Silent Speech with Ultrasound and Whisper
We trained a model to predict speech from ultrasound recordings of the tongue while speakers remain silent. Using just 50 hours of data and Whisper, our system achieves a 15.6% word error rate on open-vocabulary tasks, generalizing across new users with American accents. This approach offers a private alternative to voice commands, allowing invisible silent speech without the need for noisy indirect measurements like EMG or radar.
Just as earphones made listening private, silent speech could make speaking private.
- albert_e
This sounds promising -- would love to see this become more practical.
With wearable devices becoming more common - I am anticipating a wave of "sensors" that can be as simple as small band-aid patches that wirelessly send data to your smart device. Those sensors could also open up human-coputer-interface innovations like these.
---
In similar space ...
There was a post about a thought-to-text project from MIT no less --
"AlterEgo"
10 months ago: https://news.ycombinator.com/item?id=45174125
8 years ago: https://news.ycombinator.com/item?id=16780357
When I saw the demo posted last year it left me with an uneasy feeling -- gut feel said it was more marketing than a real working technology demo. Nothing seems to have come out of that lab since then -- strengthemning my suspicions.
- thomascountz
The use of ultrasound is interesting! The first step (identifying speech) is called subvocalization recognition, and early research (Chuck Jorgensen, et. al (2023) from NASA) use EMG[1].
[1]: https://scispace.com/pdf/sub-auditory-speech-recognition-bas...
- gblargg
I remember a story decades ago about "subvocal" speech, similar to this: https://spacenews.com/nasa-develops-system-to-computerize-si...
- hdjrudni
> The two biggest hardware challenges are reducing the size and weight of the ultrasound probe and replacing ultrasound gel with a more practical coupling material, such as hydrogel. We think both are solvable, making it possible for the probe to eventually become a lightweight wearable or adhesive patch.
Not sure I'd want to put an adhesive patch on my neck every morning so I can silently talk to an LLM in the cubicle farm. I hope this is not our future.
Very cool tech though and surprisingly good results for so little training.
I think time might be better spent improving a lip reading model (no adhesive required), assuming we're unable to read brainwaves directly.
- dschwede
Really impressive. Seems like this could help people with health problems like ALS or Parkinson's, if you can discern clear intent from their tongue muscles. Or just help people who are intubated in the hospital.