Google Phones Become Seeing Eyes for Blind

A new Google app uses a phone camera to describe the world around blind and visually impaired people. It's an artificial intelligence take on a task that's already performed remotely by human volunteers.

The app, dubbed "Google Lookout," works best when the user puts the phone around their neck like wearing a lanyard, or in a shirt pocket with the camera lens uncovered. The app then uses the camera view to analyze what's around the user and describe what it can "see".

The idea is that the app activates in situations where a blind user might ask for help, adjusting to context. For example, it can highlight the location of an object a user might reach for - such as a pair of scissors. It could also tell them when a dog is passing in front of them or read out a sign. While this is automated, users can request a description of the current "scene" by knocking on the device twice.

Dedicated Shopping Mode

The app lets users quickly switch between three modes. Explore mode is designed for general everyday tasks at home and visiting new places.

Shopping mode is optimized to find and read barcodes to quickly give the user information about a specific product. Meanwhile, Quick Read mode is designed for close-up reading such as mail, labels and signs.

There's a variety of control options including knocking on the device, holding a palm over the camera, and using the fingerprint sensor to "swipe" through modes.

Human Volunteers Provide Similar Service

To start with, the app is only available for Google's own Pixel devices in the US. That's partly because Google is confident about how it will work on those handsets, but partly as a controlled experiment on a limited number of users. It can then improve the app based on feedback before rolling it out to other devices. (Source: blog.google)

Several similar services already exist for blind users, but those apps do not harness artificial intelligence. Instead, users connect through a videolink on the phone to either trained professionals or volunteers depending on the service. The helpers act as a remote personal assistant to deal with a specific situation such as describing greetings cards in a store. (Source: ttps:.ttps:)

What's Your Opinion?

Do you think such technology can really work with artificial intelligence? Do you trust Google to adequately protect the privacy of people using this tool? Can you see any drawbacks to Google Lookout?