Google expands its accessibility-focused project Gameface to Android

Google has expanded its Project Gameface, an open-source project, to Android and can now be used to control the smartphone interface. The project, aimed at making tech devices more accessible, was first introduced during Google I/O 2023 as a hands-free gaming mouse that can be controlled using head movements and facial expressions. Keeping the functioning same, the Android version adds a virtual cursor to allow users to control their device without touching it.

In an announcement made on its developer-focused blog post, Google said, “We're open-sourcing more code for Project Gameface to help developers build Android applications to make every Android device more accessible. Through the device's camera, it seamlessly tracks facial expressions and head movements, translating them into intuitive and personalised control.” Further, the company asked developers to use the tools to add accessibility features to their apps as well.

Project Gameface collaborated with the Indian organisation Incluzza which supports people with disability. Through the help of this collaboration, the project learned how its technologies can be expanded to different use cases such as typing a message, looking for jobs, and more.

Google used MediaPipe's Face Landmarks Detection API and Android's accessibility service to create a new virtual cursor for Android devices. The cursor moves following the user's head movement after tracking it using the front camera.

While this technology has been made available on GitHub, it is now up to developers to build apps using this option to make it more accessible to users. Apple also recently introduced a new feature that uses eye-tracking to control the iPhone.