Microsoft’s New iPhone App Helps Blind People ‘See’ What’s Around Them

Microsoft has used its artificial intelligence chops to automatically add captions to your selfies and guess your age. Now, the company is applying similar tech to a more practical purpose: making everyday life easier for the visually impaired.

The company’s new Seeing AI app, which is now available for the iPhone, uses the smartphone’s camera to detect and describe nearby people, text, and objects.

The app not only recognizes a person based on his or her face, but can also relay mood based on facial expressions, estimate a person’s proximity to the user, and describe physical characteristics. In a demo video, Seeing AI provided a description of a woman smiling by saying: “28 year old female wearing glasses looking happy.”

Microsoft says Seeing AI can also identify products based on their barcode, a feature aimed at making shopping easier. The app emits a series of beeps to indicate how close the viewfinder is to the barcode, helping users align the camera properly. Similarly, when reading documents to users, Seeing AI will provide spoken hints to ensure that all corners of the paper are properly captured.

Facebook has made similar efforts in the past when it comes to leveraging its image recognition technology to benefit the visually impaired. Earlier this year, the company enhanced its automatic alt text feature to be more accurate in describing Facebook photos. The technology can now piece together what’s occurring in a scene rather than just stating the subjects of a photo.

Microsoft’s new app is available in six countries, including the United States, Canada, India, Hong Kong, New Zealand, and Singapore. The company has not yet said whether it plans to develop an Android version.