The Virtual Genius of Oculus Rift

To understand why Oculus Rift matters, it helps to know who John Carmack is. You already know his work, even if you don’t know his name: Carmack is the programmer who in the early 1990s cracked the problem of how to write a video game that takes place in three-dimensional space. He’s the reason that when you play a state-of-the-art game, you’re not leaping from platform to platform or wandering through a two-dimensional dungeon, you’re running and jumping around in proper space-time, all six axes in play, backward and forward, side to side, up and down. He’s responsible for Quake, the first true 3-D game, which begat Halo and Call of Duty and all the rest of it. Carmack did for computer games what Masaccio did for painting: he turned a plane into a space.

As such, he’s the principal architect of a medium that has generated literally billions of person–hours of entertainment over the past 20 years, and like most people who’ve started a revolution, he keeps a weather eye out for the next one. That’s how he spotted Palmer Luckey and Oculus Rift two years before Mark Zuckerberg and most of the rest of the world.

On March 26, Facebook announced that it was purchasing Oculus VR, the company Luckey started in 2012, in a deal worth $2 billion. The social-networking giant is getting top-flight engineering expertise as well as the technology behind the company’s flagship and only product, a virtual-reality headset. “Mobile is the platform of today, and now we’re also getting ready for the platforms of tomorrow,” Zuckerberg said in a press conference. “Oculus has the chance to create the most social platform ever and change the way we work, play and communicate.”

Two billion dollars is a lot of money—a head-snapping amount—for a social network to pay for a two-year-old hardware company with an ultra-nerdy name that has yet to ship a consumer-ready product and whose founder is still only 21. But what’s really surprising is that Zuckerberg is putting down a massive bet on virtual reality, which until very recently was considered not just a failure but a punch line. The Oculus deal makes for a twist ending to one of the greatest and weirdest comeback stories in the history of technology.

Palmer Luckey—the name suits him—grew up in Long Beach, Calif., the son of a housewife and a car salesman. He was a natural-born tinkerer. “Self-taught!” is how he describes himself. “Explore the world around you, take things apart, put ’em back together. You can learn a lot if you do nothing but spend your entire life in your garage working on projects or in your room reading on the Internet.” As a teenager one of Luckey’s hobbies was taking apart old video-game consoles and reassembling them inside portable cases. Another one was virtual reality.

It was an odd hobby for a person Luckey’s age because the received wisdom at the time was that VR was a failed technology. Everybody has an idea of what VR is, or what it’s supposed to be: a simulated, three-dimensional, interactive world that surrounds you completely. It’s been a staple of science-fiction classics—Neuromancer, Snow Crash, Tron, Star Trek, The Matrix—and a core component of our collective pop-cultural vision of the future for decades.

But apart from niche applications like designing cars and surveying oil fields, VR never made it to market. As Luckey puts it, “the idea existed, the will existed, the people existed, the demand existed—and the technology did not.” It baffled engineers, frustrated consumers and ate up billions of dollars of R&D money. Like flying cars and robot butlers, VR is one of those revolutions that went from wow to lame without ever actually materializing in between. Nintendo tried its hand at it in 1995 with the Virtual Boy game console and lost millions. The list of virtual-reality products that launched and then died of neglect is long.

Luckey owns most of them. He probably has the world’s most complete collection of VR headsets anywhere, more than 40 of them at last count. He bought them because he was among the very few people anywhere who still thought virtual reality was cool. Unfortunately, none of the headsets worked very well. “I didn’t start out trying to build something,” he says. “I started out trying to buy something that would do what I wanted. And it became apparent that there wasn’t anything like it.” So he started building it himself.

Luckey wasn’t the only person who still cared about virtual reality, but he almost was. There was a small community of true believers, less than a hundred, who hung out on a web forum called MTBS3D to talk about it. (MTBS stands for “meant to be seen.”) Luckey was one of them. John Carmack was another.

Carmack thought VR had potential too, in spite of all the failures, and every few years he would check in on the state of the art to see if it was usable yet. In April 2012, Carmack was tinkering with a VR headset made by Sony, and he posted about it on MTBS3D. Luckey responded. He told Carmack about his own prototype, and Carmack said he’d like to buy one. Luckey was in awe. “You cannot take money from Carmack,” he says. “It would be like if Jesus said, Give me your clothes.” He sent Carmack the prototype, his only working model, for free, via regular mail.

Luckey’s device wasn’t like other headsets. In 2012, what interest there was in VR was mostly in creating a kind of virtual cinema: you’d look in the headset and see a simulated version of a giant screen hanging in the air in front of you, and you’d watch a movie on it. Not many people did. Not only was it Skymall stuff, it was pricey—Sony’s head-mounted set costs $1,000. Luckey’s device was different. It was designed to run games and to immerse you in them. It ran fast, and its field of view was very wide: the display wrapped around to eat up your peripheral vision, putting you well and truly in another world. “That’s the only way to get any kind of immersion,” Luckey says. “I didn’t want to just have a TV you could wear.”

Carmack agreed. He adapted his latest game engine for Luckey’s headset. Two months later he took it to E3, the biggest video-gaming trade show in North America, where he announced to a startled press corps that virtual reality had finally arrived. A lot of people started asking Luckey for demos. Among them were Brendan Iribe and Nate Mitchell, both alumni of a gaming-software company called Scaleform.

“The first time I saw the Rift, it was in a hotel in Long Beach,” Mitchell says. “Basically Palmer had a bunch of circuit boards, and a bucket of cables and wires, all this stuff tangled up. He set it up, plugged it in—it took him a little while, and I was sitting there being like, Is this really going to happen? Is this going to work?” At that point Luckey’s prototype was just a box that you held up to your face, running a simulation of a bare room. But when Mitchell looked inside it, something new happened. “There was no interactivity, nothing moving,” he says. “But it gave you the sensation that, wow—there’s a world inside this little box.”

Two years later, the Oculus Rift—the dorky name is a point of nerd pride—still doesn’t look particularly futuristic. It looks like a pair of chunky ski goggles with opaque black plastic where the lenses should be. Time will tell whether it’s a gateway to a new virtual frontier, but one thing is clear already: you look weird wearing it.

But put it on anyway—it embraces your head slightly more forcefully than would be ideally comfortable—because you’ll get the rare sensation of experiencing a technology that is genuinely new. Google Glass feels like what it looks like: you put it on and think, Yup, it’s a pair of glasses with a tiny screen in one lens. Oculus Rift is different. It’s not what you expect.

The first time I tried the Rift (which seems to be winning out over Oculus as the shorthand of choice) it showed a simulation of a craggy, rocky mountainside. I turned my head experimentally, and the view changed, with no discernible lag, just as it would have in reality. Instinctively my brain started looking for the edge of the image—but it didn’t come. I kept turning until I was looking all the way behind me. There was nothing but mountain back there.

Then I looked up and watched snowflakes sift down out of a gray sky straight into my face. That’s when my brain admitted defeat. It surrendered to the illusion that it was in another world. It wasn’t going to find an edge. There were no edges. The Oculus Rift is the first visual medium that doesn’t have a frame around it.

Another demo put me in the driver’s seat of an old-fashioned race car. Just sitting there, without even starting the engine, was a revelation. I leaned over and stuck my head out the window and admired the car’s exposed left front wheel assembly. If I leaned in to the dashboard I could read the fine print on the gauges. When you’re in the Rift you become the camera. You control the point of view with your body, the way you would in reality.

The Oculus Rift has limitations. The resolution isn’t high enough yet, so you have a slight sense that you’re viewing the world from inside a screened-in porch. Look down and you’ll notice that something’s missing: your entire body. Oculus can bring your eyes and, with headphones, your ears into the virtual world, but nothing else. You haunt the virtual world as a floating, disembodied spirit.

And yet it’s convincing. It’s visceral. VR offers a new kind of illusion. There’s a name for it in the industry, this deep and abiding conviction that you’re somewhere else: presence. I tried a simulation of a dogfight in outer space, and when my one-man fighter was shot out of the mother ship into the cold black void, my stomach dropped through the floor. After South by Southwest this year, a viral video circulated of the actress Maisie Williams trying an Oculus Rift simulation of the 700-ft. wall of ice in Game of Thrones. She’s standing on solid ground, but she has a full-blown panic attack—she’s afraid of heights. The illusion of being on a cliff edge is tenacious. “You can’t do that on a TV monitor,” Mitchell says. “You can’t do that on a phone. You’ve never been able to do that before in the history of humankind. You know you’re not going to fall, but your brain’s saying, Don’t take that step.”

Two years after he mailed his prototype headset to John Carmack, Palmer Luckey is somewhere else: a black glass office tower in Irvine, Calif., the headquarters of Oculus VR, where he now has the title of founder. Carmack, 43, is his CTO. After 22 years he quit his job at Id Software, the company he co-founded, to work at Oculus. Brendan Iribe is now Oculus’ ebullient, hyperverbal CEO. He left behind unvested options from his last company to come to Oculus. It has that effect on people.

Why could Oculus solve VR when nobody else could? The answer takes some explaining. VR presents an intractable mass of interconnected engineering challenges, most of which start with your brain. The problem with your brain is that it’s smart, and it’s difficult to fool. The human brain is constantly taking in data about the world. Some of it comes in through your eyes; some of it comes from your vestibular system, your inner ear, which provides your sense of balance and orientation. Your brain’s constantly cross-checking those data sources to make sure they match up. If they don’t, bad things happen.

Say, for example, you’re wearing a virtual–reality headset that is telling your eyes that you’re on Mars. If you move your head, the view of Mars has to change too—instantly, with no latency, the way it would in reality. If it doesn’t, your eyes get out of sync with your inner ear. Even a delay of 50 milliseconds between head-turn and view-change is too much. Your brain will spot it.

In fact, it’ll get really upset about it. So much so that it makes people feel nauseated—it’s one cause of a phenomenon known as simulator sickness, which is similar to motion sickness. Individuals’ tolerance for latency varies, but at Oculus they peg the maximum allowable lag at 20 milliseconds. On a technical level, that’s a challenging specification to hit. By comparison, an eyeblink takes about 300 milliseconds.

A headset also has to deliver new frames to the eye absurdly fast in order to keep the image from smearing or freezing when you move. It has to have two tiny high-definition monitors in it, one for each eye, and they have to cover a field of view wide enough that it blankets your peripheral vision. It has to be simple enough to mass-produce and cheap enough that people can afford it. It has to be light enough that it doesn’t hurt your nose.

Getting this kind of precision requires tight integration of hardware and software—it’s one of the mantras you hear around the Oculus offices. And beyond that, it takes a solid grasp of the fundamentals of gaming technology. That’s where a guy like Carmack, who invented some of the technology in question, comes in handy. “The science around this is so close to the metal,” Iribe says. “It’s so close to what bits are happening when. Carmack knows he can go in and get that fully optimized.”

Oculus began on Aug. 2, 2012, with a campaign on Kickstarter. The goal was to raise $250,000; the project passed that figure in two hours. By the time the campaign closed 30 days later, backers had pledged $2,437,429. Since then Oculus has taken 75,000 orders for its development kit, which is a nonfinal, prerelease version of the headset intended primarily as a tool for people who want to write software and develop content for it. In December it closed a $75 million round of financing from venture-capital firm Andreessen Horowitz. Then, of course, came the Facebook purchase.

Not even the founders saw it coming, or not at first. Zuckerberg first met Iribe last November. “He came down,” Iribe recalls, “and we showed him some of the internal prototypes, and he got so excited about the vision of what we were doing and about the potential that this is truly the next computing platform. He actually said that to us. And it’s like, ‘Wow! We are looking at this whole thing being just that gaming platform. But tell us more, Mark.’ And he started to describe it, and we started to believe it too. And we started to relate it to a lot of the experiences we were having.”

It had been dawning on Luckey and Iribe and their colleagues for some time that they might not be as clear as they thought they were on what virtual reality is actually for. It began as a gaming technology, but it turned out first-person shooters weren’t the killer app they expected. “Pretty quickly we realized, ‘O.K., maybe running down hallways at 40 m.p.h. isn’t exactly the most comfortable thing to do in VR when you’re sitting in a chair,’” Iribe says. “As we started to build these made-for-VR experiences, we started to realize that intense gaming, where there are bullets flying at your head, can be actually a little too intense.”

So they started thinking more broadly about what exactly it was they were building. Iribe mentions virtual vacations and a 3-D VR encyclopedia as future possibilities. Mitchell describes a “magic school bus” that could take a bunch of kids on an instant field trip to Florence to look at Michelangelo’s David. But the really big opportunity, the mainstream, billion-user opportunity, was in virtual reality as a next-next-generation communications medium. “When you add other people to it,” Iribe says, “and you can actually see somebody in that place and you can make eye contact, and you can look at them and they can look around, you can now have this shared sense of presence in this new gaming experience, entertainment experience or just social experience that really starts to define what virtual reality is all about.”

The news that Facebook was acquiring Oculus was not received with universal happiness in the gaming community that had backed the company in the first place. The announcement on Oculus’ blog quickly grew a comment trail 900-plus posts long essentially arguing, in various ways, that Oculus had abandoned its hardcore hacker roots to become a bland, corporate, three-dimensional ad-serving platform. Markus Persson, the creator of Minecraft, was an early backer, and he visited the Oculus offices earlier this year. He summed up the attitude when he tweeted to his 1.54 million followers, “We were in talks about maybe bringing a version of Minecraft to Oculus. I just canceled that deal. Facebook creeps me out.”

Luckey is quick, very quick, to assert that this isn’t a pivot away from gaming and toward something else. “Nope,” he says. “No pivot. We’re doing what we’ve always done. We’re continuing to operate independently, and if anything, we’re putting more resources into games, not less. This lets us invest in content, make better tools for content, better developer relations, and build a much better platform for games.” Iribe is right behind him: “People have not even seen our final form. There are so many cool things that happened directly because of this deal. It’s one thing to have an initial first impression of a deal that might not make sense on the outside. It’s another to see the proof of it once the big picture becomes clear.”

Iribe points out one concrete benefit for users: cheaper headsets. Now Oculus can afford to sell them at cost. “It changes our priorities from making money to making virtual reality happen.” Iribe rejects the idea that he and his colleagues sold out. “If anything, I think Facebook got an incredibly good deal,” he says. “If we stayed independent, we could probably have made a lot more.” Brian Blau, a consumer technology analyst at research firm Gartner, says, “They want to seed the market. They want to get it in front of more developers and more early adopters. And that’s the way to do it, to give it away as cheaply as they can.”

Zuckerberg clearly has a lot of faith in the Oculus team, because as far as they’ve come, there are a lot of technical challenges left to solve before virtual reality can become a social medium at all. It will have to track more than your head: it’ll have to track your hands, your mouth, your facial expressions, your gaze. That’s not part of the existing technology. At the moment virtual reality is still a pretty lonely place.

It will also have to morph into a form factor that nontechnophiles will be willing to put on their faces. And it will face competition. Earlier this month Sony unveiled a new VR headset of its own, with the working name Project Morpheus. It will presumably connect to its popular Playstation 4 console, which already has millions of users.

For the next few years at least, Oculus VR is going to be what it started out as: a high-end gaming peripheral, supplemented with content from adventurous creatives in the broader entertainment world. “We’re working a lot with people who want to do things like immersive movies or music videos or meditation or relaxation applications,” Iribe says. “It’s kind of like the beginning of film. It’s going to take this whole new set of mechanics and engineering to master it. We have no idea what really works in VR. People ask us, What’s the holy-grail app going to be? I have no idea! Don’t know.” The uncertainty doesn’t bother him.

For now, Luckey and Carmack and the rest of them are still poised at the crest of the wave. Their money worries are over. Now they just have to safeguard what made Oculus so exciting in the first place, back when it was just a box with a room inside it. “I think people have always wanted to experience the impossible,” Luckey says. “That’s one of the reasons games have caught on. They want to actually do things themselves, have a say in how that world works, instead of just watching someone else do it.”

With reporting by Victor Luckerson

Contact us at letters@time.com.