Speech-to-Text System using Augmented Reality for People with Hearing Deficits
| Reference number | |
| Coordinator | RISE Research Institutes of Sweden AB - RISE Research Institute of Sweden, Kista |
| Funding from Vinnova | SEK 1 304 000 |
| Project duration | November 2019 - February 2021 |
| Status | Completed |
| Venture | Digital health |
Important results from the project
The purpose of the project was to study the visual, optical and ergonomic conditions for a speech-to-text system that uses text for Augmented Reality (English: "Speech-to-Text System using Augmented Reality", abbreviated STAR) to present text for people with impaired hearing. The purpose and goals of the project have largely been met, but further technical development of a demonstrator is planned, as well as widening towards other sensory impairments e.g. of vision.
Expected long term effects
1. Shown how readability and legibility can be studied in AR and VR. 2. Provided instructions on font design for AR glasses. 3. Studied visual search behavior in AR. 4. Provided instructions on design of text presentation in a VR. 5. Shown possible connections between eye movements and presentation of text in VR. 6. Studied how the eye´s search behavior can occur in VR. 7. Shown how a future STAR glasses for the hearing impaired can be designed. 8. Measured luminance and contrast for AR glasses. 9. Components for a demonstrator procured. 10. Market study on speech-to-text systems conducted.
Approach and implementation
Re-prioritizations and improvisations were necessary due to the Pandemic. Optical lab was set-up in a wardrobe of an employee. User studies were conducted in a room in Uppsala intended for students and in an apartment. Hearing impaired persons could not be used in the test, so normal hearing persons were used. Tests in AR were performed with different reading tasks. Legibility was measured in VR glasses along with eye movements. Many important results have been achieved, even if not all goals were met. A good basis for producing a demonstrator has therefore been created.