The Unlikely Secret Making Cars Much Smarter

Driverless cars, which take passengers from A to B with minimal human input, are already hitting American roads. A variety of automakers and technology firms are experimenting with driverless technology, ranging from entirely driver-free vehicles to driver-assist functions like automatic lane following.

What makes much of this possible is an array of sensors and cameras, which absorb data about the environment around them. Those sensors can’t take action on what they see without help, though—they need a brain in the form of a powerful onboard computer.

In the future, this brain may be supplied by Santa Clara, Calif.-based NVIDIA, best-known as a purveyor of high-end graphics cards for personal computers. The company says innovations in gaming graphics are well-suited to the needs of autonomous vehicles. Danny Shapiro, NVIDIA’s senior director of automotive, told TIME why. What follows is a lightly edited and condensed transcript:

TIME: Let’s start at the beginning: How did NVIDIA first get into cars?

Shapiro: We started as a company over 20 years ago. The focus initially was just in doing 3D graphics for computer gaming. What we’ve been able to do is take the technology from gaming and look at other markets where that could be introduced. So, our company is focused on visual computing.

About a decade ago, we had some automotive customers who wanted to increase the level of visual fidelity and really improve the experience inside the car. That’s when you started to see screens inside cars that displayed different kinds of graphics. And so we started taking graphics processors that were in laptop computers to drive the screens in the cars. Today, we have over 8 million cars on the road with NVIDIA technology inside, ranging from Audis to BMWs to Hondas and Minis. Tesla Motors is one of our flagship customers, with their 17-inch touchscreen.

So when did NVIDIA make the jump from powering vehicles’ screens to enabling automated driving?

Graphics processors are not just used for graphics. If you look at data centers now, high performance computing has turned to the GPU (graphics processor unit). Massive number crunching, parallel processing, everything from natural language processing to people like Shazam to Facebook doing image recognition, all these kinds of high performance computing tasks are happening on the GPU. And so we’re able to do the same kinds of very advanced processing in the car as well.

Sensors on vehicles are generating massive amounts of data. You have cameras, radar, LIDAR. These are all pretty much dumb sensors. There needs to be a very powerful brain that’s taking that data and making sense of it. That’s where we come in. We developed a platform for vehicles, we call it DRIVE PX, that’s our autopilot computer.

How powerful does that computer need to be?

I’m still blown away by the fact that this system is really bringing true supercomputing horsepower into the car. There’s a chip that we brought out earlier this year called Tegra X1. Its processor delivers over 1 teraflop of performance. If we go back to the year 2000, our country’s fastest supercomputer was 1 teraflop. It took up a room that was 1,400 square feet. We have a chip the size of a thumbnail.

Though 1 teraflop of performance, we can certainly render beautiful graphics, but we’re also able to handle the gigabytes of data that are being generated by all these sensors and we’re able to make sense of them. We’re able to identify pedestrians, or recognize other vehicles. We can take the radar data and fuse it with camera information, what’s called sensor fusion, to increase the accuracy of these systems.

Basically if you’ve seen any of the self-driving prototypes, whether it’s Audi or Mercedes or BMW, if you open up the trunk, it’s just full of PCs. With this automotive solution that we’ve developed, we’re bringing supercomputing horsepower into a mobile form factor. We’ve developed a system that can be hidden in the trunk of the vehicle.

Back to the Future’s DeLorean Isn’t the Only Awesome Retro Car

Does your work in PC graphics inform your auto products, or vice-versa?

It really is bidirectional. We have a core architecture platform that everything is built on. We then make products that are serving different industries, and it scales. Our high-end graphics processors that a gamer would put in their PC would cost $1,000 and have 3,000 cores. For the automotive space, we needed to have something that’s extremely compact and energy efficient. So we leveraged that architecture, but the one in the car may only have 256 cores.

For car buyers, is in-vehicle tech now more important than horsepower and torque?

No question. Certainly design aesthetic and quality are very important, but that’s just the minimum bar. You have a class of car buyers that are very tied to their mobile devices, and want to pair their phones via Bluetooth. In some cases that’s make or break. If your phone doesn’t work with this car, you don’t want to buy it.

For example, we worked with Audi to bring Google Earth and Street View into the car. That became a huge selling feature for them. It’s also a safety feature. You can actually tell the car, “I’m hungry for sushi,” and it’ll bring up a list of Japanese restaurants on the screen, and you’ll be able to see the 360 panorama of that storefront before you even start driving. Then it becomes a lot safer to navigate there, rather than looking down at your phone and looking for street numbers.

The challenge has been that until somebody experiences some of these things, whether it’s that kind of navigation experience, or blind spot monitoring, or forward collision alerts, you don’t know what you’re missing. But once you have that or see that in a car, you go, “I don’t want to buy anything without that.”

PC sales keep declining year-over-year. How important is the automotive space for NVIDIA’s future?

Right now, it’s a relatively small part of NVIDIA’s total revenue. But it’s the fastest-growing part of NVIDIA. It’s something we’re investing in very significantly. Last quarter we grew 85% year-over year. It’s something that our CEO is very involved in. It’s a huge growth opportunity for us. We have 8 million cars on the road today; we expect to have over another 25 million cars over the next four years.

In your eyes, what’s the coolest thing on the horizon for the auto world?

Autonomous driving is definitely cool. I’ve had the fortune to ride in autonomous cars from Audi, from BMW, from Tesla, and it’s a huge thrill. I love to drive, and the Audi was actually on the race track, so I was going around at 150mph and no one was touching anything in the car while I was doing laps on the track.

The promise here is taking away the driving situations where people don’t want to drive. They’re stuck in traffic, they’re on the highway, they’re trying to find a place to park. So to be able to get out of your car at the movie theater and say “go park yourself,” and the car will just drive itself, identify an open spot and park and wait, and then you call it back with your phone. It’s about great time-saving efficiencies. If you can get work done while you’re commuting, or in the future be able to sleep while driving down to some other destination and just have the car drive itself while you slept, that would be great.

A really great benefit no one talks about and it’s true is the reduction in accidents, injuries and fatalities. It’s going to be a huge life saver once we’re able to get these technologies on the road.