- Lyu Theresa
The link is to my project demo on youtube.
It includes a ~8 minutes ted talk on the second slide in the original demo which was block by youtube due to the copyright. Sorry for the inconvenience.
top of page
Search
- Lyu Theresa
The applications of AlterEgo can be classified under three broad categories:
Closed-loop Interface
The device enable users to silent speech interface with a computing device where certain computer application responds to subvocal queries with aural feedback, thereby enabling a closed-loop, silent and seamless conversation with a computing device.
For example, the AlterEgo system allows users to internally vocalized an arithmetic expression like 3 plus 5, the application would output the answer 8 to the user, through bone conduction headphones.
Moreover, the device can also be used as a reminder and scheduler at specific times, which will be aurally output to the user at corresponding times, thereby providing a form of memory augmentation to the user.
The silently communicating can also be used in artificial intelligence, act as an adjunct to human cognition in a personalized manner. The researchers implemented human-AI collaborative chess and Go through bidirectional silent speech, where the user would silently deliver the game state and AI compute then aurally output the next move to be played.
Open-loop Interface
The device could also be used as single silent speech input to applications, to control devices and to avail services.
An example is to connect to a smart home controller, the device enables the
user to control home appliances and devices, switch on and off, television control, home lighting etc through internal speech, without any observable action.
The interface can be personally trained to recognize phrases meant to access specific services. As an example, internally vocalized 'Uber' to the home could be used to book transport from the user’s current destination using the interface.
Human-human interface
The device also augments the way how people communicate. It can be used as a back-channel to communicate with other people or even a public session in the future.
In the current version of the device, the user can internally communicate 5 common conversational phrases to another person through the interface Figure. \ref{figure:data_corpus}. This interface could be expanded with further training since the researchers have created an environment for the device, which allows for peripheral devices to be directly interfaced with the system. For instance, smart-glasses could directly communicate with the device and provide contextual information to and from the device.
- Lyu Theresa
To evaluate the robustness of the platform, 10 participants (6 female, average age 23.6 years old) were invited in the evaluation experiment. None of them has experienced the same study before, to collect the unseen data. The arithmetic computation application is used as the basis for accuracy evaluation.
For each participant, showed them a total of 750 digits, randomly sequenced on a computer
screen, and instructed the users to read the number to themselves, without producing a sound and moving their lips. The digits were randomly picked from digits 0 to 9, such that each digit exactly appeared 75 times. The data was recorded for each user with the trigger voltages marking word-label alignments.
The experiment was conducted in two phases. First, collect silent speech data from users and evaluated word accuracy on a split training test. Second, assess the recognition latency of the interface by testing live inputs on the model, trained using the previous step.
bottom of page