Possibilities with AR Glasses

Technology can help enhance the communication abilities of the deaf or hard-of-hearing persons and make it easier to navigate and participate in the world around them. If you have heard of AR glasses, it has the potential to greatly improve the lives of people with hearing loss as an accessible tool to visualize sounds.

There are now AI-powered smart glasses that allow deaf people to participate in real-time conversations by displaying subtitles on the glasses.

The project uses a combination of speech and sound AI to visualize speech and non-speech sounds. The visualizations are then overlaid onto AR glasses, making it possible for users to see and interpret sounds that they wouldn't be able to hear otherwise.

Audio can include a wide range of sounds, from human speech to non-speech sounds like barking dogs and sirens. When designing accessible applications for people with hearing difficulties, the application should be able to recognize sounds and understand speech.

Combining speech and sound AI together, you can overlay the visualizations onto AR glasses, making it possible for users to see and interpret sounds that they wouldn’t be able to hear otherwise. They would be able to understand their surroundings better, and they would be able to avoid potential hazards.

In this scenario, automatic speech recognition (ASR) is used to enable the glasses to recognize and understand human speech. This technology can be integrated into the glasses in several ways:

# Using a microphone to capture the speech of a person talking to a deaf or hard-of-hearing individual and then using ASR algorithms to interpret and transcribe the speech into text. This text can then be displayed on the glasses, enabling the deaf or hard-of-hearing person to read and understand the speech.

# ASR can also be used to enable the glasses to respond to voice commands so that users can control the glasses with their voice.

# They are also able to display all conversations on the screen, such as transcribing voice directions from maps while you drive and any other sounds like horns or sirens from emergency vehicles and wind noise.

According to the World Health Organization, about 1.5B people (nearly 20% of the global population) live with hearing loss. This number could rise to 2.5B by 2050.

Cochl, an NVIDIA partner based in San Jose, is a deep-tech startup that uses sound AI technology to understand any type of audio. They are also a member of the NVIDIA Inception Program, which helps startups build their solutions faster by providing access to cutting-edge technology and NVIDIA experts.

The platform can recognize 37 environmental sounds, and the company went one step further by adding cutting-edge speech-to-text technology. This gives a truly complete understanding of the world of sound.

S Mohini Ratna, Editor, VARINDIA