Home Apps Microsoft’s ‘Seeing AI’ app lets your iPhone narrate what it sees

Microsoft’s ‘Seeing AI’ app lets your iPhone narrate what it sees

Microsoft is releasing a new app — in the US only for now — that “sees” for you and narrates the world around you.

Seeing AI. It's a new app from Microsoft designed for the "low vision community", and is a research project app that “harnesses the power of AI to describe people, text and objects”.

Sadly, it is only avaialble in the US at the moment to iPhone users, but hopefully will become more widely available.

If you saw the movie ‘Her’, which featured the voice of Scarlett Johansson as the artificial intelligence in the protagonist’s computer and phone, you can imagine that AI would have used similar technology to understand what its camera saw, except at a vastly more advanced level, which is natural for a sci-fi film.

So, what kinds of things does Microsoft say Seeing AI can do?

Short Text
Speaks text as soon as it appears in front of the camera

Provides audio guidance to capture a printed page, and recognises the text, along with its original formatting

Gives audio beeps to help locate barcodes and then scans them to identify products

Recognises friends and describes people around you, including their emotions

An experimental feature to describe the scene around you

Identify currency bills when paying with cash. (Coming soon, US currency only for now)

Turns the visual world into an audible experience.

Now, given that the Google Search app lets you look at things and can tell you, for example, what kind of plant you’re looking at, which is something that Samsung's Bixby app for Galay S8 and S8+ can do too, this tech is definitely on its way for all, and a great example of how far artificial intelligence is coming.

Thankfully a completely autonomous and fully artificially intelligent Terminator robot with the same technology to identify humans is still in the realms of science fiction – or at least, for now!

Also - if you create a free iTunes US account (there are instructions online if you search), it is actually possible to download the Seeing AI app in Australia, but then when updates are due, you'll have to flick between Apple ID logins for the iTunes store, which can be annoying - but it can be done.

Here’s the video:


Did you know: 1 in 10 mobile services in Australia use an MVNO, as more consumers are turning away from the big 3 providers?

The Australian mobile landscape is changing, and you can take advantage of it.

Any business can grow its brand (and revenue) by adding mobile services to their product range.

From telcos to supermarkets, see who’s found success and learn how they did it in the free report ‘Rise of the MVNOs’.

This free report shows you how to become a successful MVNO:

· Track recent MVNO market trends
· See who’s found success with mobile
· Find out the secret to how they did it
· Learn how to launch your own MVNO service


Alex Zaharov-Reutt

One of Australia’s best-known technology journalists and consumer tech experts, Alex has appeared in his capacity as technology expert on all of Australia’s free-to-air and pay TV networks on all the major news and current affairs programs, on commercial and public radio, and technology, lifestyle and reality TV shows. Visit Alex at Twitter here.