This Is Augmented and Mixed Reality’s Essential Missing Ingredient

The tech world is buzzing over three new technologies on the horizon, and industry vanguards are placing billions of dollars in investment bets.

The first one, virtual reality, hit the scene nearly four years ago, when Oculus VR unveiled an advanced prototype of its Oculus Rift virtual reality headset at the Consumer Electronics Show in January 2014. It became the trade event’s biggest hit, and shortly thereafter, in March 2014, Facebook bought Oculus VR for $2 billion.

Not long after, Microsoft introduced its HoloLens project. It was distinct from Oculus VR’s notion of an audiovisual wraparound experience that enclosed wearers in immersive virtual worlds. Instead, Microsoft’s HoloLens smartglasses allowed you to view the real world through its lenses, then superimposed virtual images or objects on the scene — a hybridized approach known as augmented reality, or AR.

And if you’ve been watching this space recently, you’ve probably heard talk of a third technology called mixed reality, or MR. It’s similar to augmented reality in that it mixes virtual with real world scenes, but does so in more concrete ways. Mixed reality is essentially an attempt to bridge the gap between VR and AR. It keeps you grounded firmly in the real world, but with objects or images so lifelike and convincingly meshed with reality that it’s difficult to distinguish the imaginary from the real.

Most people got a glimpse of AR, with its virtual objects displayed through smartphone screens, when studio Niantic unleashed Pokémon Go on the world in July 2016. The game, based on The Pokémon Company’s popular 20-year-old creature-snatching franchise, placed virtual monsters in real world scenarios using a smartphone’s camera, then tasked players with capturing them. It was like discovering one world hidden beneath another, easily accessible through a gadget people already owned. Paired with Pokémon’s formidable cultural cachet, Pokémon Go deserves credit for turning AR into a household name.

Since then, Apple has created a powerful mobile AR suite of tools with ARKit for developers, and hundreds of AR apps are already available on iPhones and iPads. Google has also jumped into the AR game with ARCore, its own suite of AR developer tools for the Android platform. In both cases, these AR apps are delivered through smartphones or tablets. Which raises the question: is AR’s broadest audience going to experience the technology on mobile devices (like smartphones and tablets), or through special eyewear that lets them view and interact with AR and mixed reality applications without hoisting a device?

Niantic chief executive John Hanke had an answer at a recent Wall Street Journal conference, discussing the success of Pokémon Go and making an important prediction.

Speaking about glasses, Hanke said he thought it would take “probably in the order of five years” for AR to go mainstream. Augmented reality debuted on smartphones, said Hanke, “because you build it for the platform that exists.” But Hanke believes AR won’t reach “full fruition” until we’ve transitioned to glasses, at which point the technology can be fully “woven into your daily life.”

In my 35 years analyzing and covering Silicon Valley, I’ve learned that when the pioneers of a technology weigh in on something they’ve been involved with, they’re engaged in more than mere hyperbole. Mr. Hanke’s studio has clearly been a pioneer in AR, and millions of people have played Pokémon Go. As he states in the Journal article, he created the game for a platform that already existed — a platform without which the technology wouldn’t have flourished. But it’s also clear he believes AR won’t reach its potential until we’ve shifted from a platform that’s not in our permanent sightline, to one that is.

On the other hand, Apple CEO Tim Cook is over the moon about AR for the iPhone. In multiple interviews, he’s said how excited he is for AR, and that he believes the technology is a game changer for the company’s mobile platforms. He’s thus committed Apple to working closely with developers to create the most innovative AR apps possible using ARKit for iOS. Google seems to be as excited about AR on its Android smartphones, though the company’s leadership hasn’t been as vocal about the technology as Apple’s Cook. No doubt it’s also proceeding more cautiously after the disaster it ran into with its Google Glasses eyewear a few years back.

Regardless of who’s more committed or ultimately more successful, we’re in the gateway phase of an emerging and ineluctable technological sea change. AR on smartphones and tablets is about acclimating people to concepts like AR and MR. When the forces of technological innovation and miniaturization finally converge in a way that yields mass market glasses or goggles no one’s embarrassed to wear, we’ll have today’s devices to thank in part for paving the way and establishing demand.

When will that be? You can bet Apple, Google, Microsoft as well as plenty of others are working flat out to come up with eyewear mainstream users won’t balk at, and all have filed multiple patents on various glass designs. But as Niantic’s Hanke says, it’ll be at least five years before someone reintroduces glasses of a sort that brings AR to the masses in a more personal and interactive way.

I’m excited about AR on smartphones, but agree wholeheartedly with Hanke. The sort of AR that’s a constant companion, that’s teased in so many Hollywood science fiction flicks, is still at least half a decade away, if not more. In the meantime, odds are we’re probably on the verge of experiencing more head-turning, industry-upending AR apps on our smartphones and tablets. Just keep in mind as you do, that they’re stepping stones, and that we’ll eventually need AR glasses for augmented and mixed reality to reach its vast and untapped potential.

Tim Bajarin is recognized as one of the leading industry consultants, analysts and futurists, covering the field of personal computers and consumer technology. Mr. Bajarin is the President of Creative Strategies, Inc and has been with the company since 1981 where he has served as a consultant providing analysis to most of the leading hardware and software vendors in the industry.