The Cloud Will Be the Key to Our Robotic Future

Robots haven’t yet lived up to their potential, but they’re about to benefit enormously from the Cloud. I’ll describe how a new generation of robots can use wireless networking, big data, machine learning, open-source, and the Internet of Things to improve how they assist us in tasks from driving to housekeeping to surgery.

The roots of this trend go back to the early 1990s, when the World Wide Web was first introduced. I was a young professor at the University of Southern California directing my grad students in a robotics research lab. One afternoon they came to my office and showed me something that blew my mind. On my desktop computer, they launched Mosaic, the first web browser. We explored some of the first web sites, including one where anyone could view a live camera pointed at a coffee pot in a college student lounge. The camera was rigged up by Cambridge University grad students to check when there was a fresh pot available; what you might call “disruptive caffeination.”

That night, my students and I stayed up late in the lab brainstorming about how we could take that idea further. Instead of passively watching things with webcams, could we use the Web to let remote visitors actively move and change the environment in our lab using our robot?

Rather than having the robot do something boring like stacking blocks, what if it could tend a garden filled with living plants? We took an industrial robot arm and fitted it with a digital camera, an irrigation system, and a pneumatic nozzle to pick up seeds. We installed it at the center of a custom circular aluminum planter three meters across and filled it a half-meter deep with potting soil and some starter plants. We designed a graphical web interface that anyone could access to view, water, and plant seeds in the garden by moving the robot.

The Telegarden went online in the summer of 1995. Word got around, and within weeks, thousands of people were visiting this community garden. Many came back regularly to water their plants. There was a chat room where people would post requests saying they were going on vacation and asking if someone could water their plants. Thousands of seedlings began to sprout, and the Telegarden quickly became overgrown. It turned into a study of the Tragedy of the Commons.

We were surprised; we’d been concerned that gardening might be the last thing people would want to do online (this was ten years before Farmville). After a year we were invited to install the Telegarden in a museum in Austria, where it remained online, 24 hours a day, for nine years.

To our knowledge, more people operated that robot than any robot in history.

The Telegarden was the first active device on the web, MIT Press published two books about it, and soon many other devices and systems were connected to the web.

Since then, the field of Robotics has advanced considerably. There are now hundreds of research labs and over a dozen journals. There are over 5 million service robots like the Roomba vacuuming homes and offices and over 3000 robots assisting surgeons in operating rooms around the world. There have also been major advances in digital cameras, inertial motion sensors, and other sensors. When Microsoft introduced the Kinect 3D camera for gaming, it was a major breakthrough for robotics, providing a low-cost way to obtain 3D point clouds that can help robots navigate and manipulate. In 2012, President Obama announced the U.S. National Robotics Initiative with over $70 million in new funding for research.

But robots are not yet folding our laundry or loading our dishes into the dishwasher. These mundane chores are fiendishly difficult for robots. The essential problem is uncertainty. Put yourself in the position of being a robot: everything around you is blurry and unsteady, low-resolution and jittery, you can’t really tell what things are, where they are, and how they are moving. You can’t perfectly control your own hands, it’s like you’re wearing huge oven mitts and goggles smeared with Vaseline. I hope you’re having some sympathy now for robots.

I believe the Cloud is the key to a new generation of robots. Take Google’s robot car. The car uses the network to access Google’s enormous database of maps and satellite and Streetview images and combines it with streaming data from GPS, cameras, and 3D sensors to monitor its own position within centimeters, and with past and current traffic patterns to avoid collisions. This gives Google an enormous advantage over automobile companies like Toyota and General Motors.

Why is Google interested in Robots? Because Google understands the Internet.

BTW, I like Brad Templeton’s observation: “A robot will be truly autonomous when you instruct it to go to work and it decides to go to the beach instead.”

Although robots have been on the Internet for 20 years, in 2010 James Kuffner, a brilliant researcher at Google, coined the term “Cloud Robotics.” The Cloud isn’t just a new name for the Internet. It’s a new paradigm that uses the Internet in new ways. Think of Google Docs. Anyone can send Microsoft Word documents over the Internet, but Google Docs is different: The document and the software don’t live on your computer. Everything is stored in the Cloud using remote server farms with shared memory and processors. This is helpful because you don’t have to worry about your disk crashing or maintaining and updating your software or hardware. The Cloud also provides economies of scale and makes sharing data across applications and users easier than ever. (Of course, it also raises huge privacy and security concerns.)

Here’s what I see as the Five Elements of Cloud Robotics

1. The first element concerns memory. The 2012 indie film Robot & Frank is a masterpiece that offers a unique and I think quite realistic glimpse into the future (like the 2013 film Her by Spike Jonze). A man is growing older and becoming forgetful. His kids send him a robot to help him around the house. It cleans up for him and reminds him to eat healthy and to water his garden. When I saw this film, I could imagine myself wanting a robot like this to remind me to eat kale, take my meds, and do my situps, but also to keep me company, reminding me of relevant memories, maybe even telling me jokes based on current events and what’s happening in my environment. But such household robots aren’t available yet. One problem is that there are thousands of objects in a typical house.

Consider designing a robot to declutter your house. This is very important for anyone who has kids and especially for those of us who are getting older. When a senior citizen drops something on the floor, they may not notice it because of poor eyesight. Even if they see it, it’s not easy to reach down and pick all these things up. But the consequence of slipping and falling can be catastrophic; if you break your hip, for example, it could lead to being bedridden, which leads to loss of circulation and depression. After a certain age, things left on the floor can be fatal.

What if a robot could work quietly while you are sleeping or at work, picking things up off the floor and putting them where they belong? The problem is that no matter how well you program the robot, no matter how many objects it stores in its onboard memory, there will always be something that it hasn’t seen before. Like this new remote control I bought to advance my slides. If it fell on the floor, my robot may not know what to do with it. Is it a chocolate bar? Does it belong on my desk? In the fridge? Or in the garbage?

Fortunately, any robot that’s working in your house will be connected to a wifi network. So it has access to a vast library of information on the Internet, where there’s information on almost every object imaginable. It’s an enormous amount of information, and it’s constantly growing. The problem is that keeping all that memory onboard the robot in your house isn’t feasible. But the Cloud makes that information available on demand. The first element of Cloud Robotics is Big Data.

2. The second element of Cloud Robotics addresses limitations in onboard computer processing. Robots can carry at most a few computers, and there are many problems that require far more computation than those can provide.

Robots are starting to use a statistical approach known as Belief Space.

One exciting approach is to model the environment, sensors, and actions using probability distributions. The mathematical term for this is Belief Space. I know that sounds like something spiritual or from science fiction. But it’s shorthand for Partially Observed Markov Decision Processes (POMDPs). To solve them, to find the optimal action in this context, requires taking the convolution of several distributions. This quickly becomes infeasible as the probability distributions become more complex, multi-modal, and non-parametric.

Finding solutions requires an enormous amount of computing power. Belief Space was considered intractable until very recently, when we gained access to clusters of computers on demand through the Cloud. Such computing power also facilitates statistical optimization, machine learning, and planning motions in high dimensions with many simultaneously moving robots. The second element of Cloud Robotics is Cloud Computing.

3. The third element of Cloud Robotics uses the fact that humans are increasingly connected over the web, exchanging information with each other. Here’s an example. I was born in Nigeria, and I went back there a few years ago. I was surprised to find that there was an enormous interest among students in robotics. In Africa, like all countries, when kids they see a robot, it’s very physical, it’s something that they can identify with, and it catches their interest. Robots are a “gateway drug” for getting students interested in Science, Technology, Engineering, and Math (STEM). I met students in different areas of Ghana that were excited about robots, but in many cases they didn’t know about each other.

Robots are a “gateway drug” for getting students interested in Science, Technology, Engineering, and Math.

I met a wonderful professor, Ayorkor Korseh, at Ashesi University. We decided to start the African Robotics Network. And the idea was that we wanted to connect all these groups around not only Ghana, but all over Africa, who are interested in robots, and allow them to learn from each other and develop new tools and new ideas.

Existing educational robots are still relatively expensive. We saw the need for an ultra-affordable robot for education. We created a website and announced a worldwide Design Competition. We set the price target at US $10. We figured that was completely ludicrous but it would get people thinking. After four months we received 28 submissions. Many elegant designs: but the grand prize winner really surprised us.

It was a robot that was made out of a Sony game controller. A designer named Tom Tilley modified a game controller, which anyone can get surplus for about $3 or $4. There are two motors that are built into it that create vibration as you are playing the game. He took them out and attached two wheels to them. The body of the game controller became the body of a mobile robot. He also added lights, but he wanted something that would detect when it bumped into something.

He realized that he could use the two thumb switches on the top. The problem was that when he tested it, the thumb switches wouldn’t react; they needed more leverage. So he needed a counterweight for the thumb switches. He thought about what might work that would be available on fairly low cost and came up with a brilliant idea: lollipops.

What kid could resist a robot with two lollipops on top of it?

Tom calls it the “Lollibot,” and you can find detailed instructions on how to make your own on the web. The total price for all the parts comes to $8.96 (and that includes the lollipops)!

Another exciting development is ROS, the Robot Operating System. It’s an open source software library that has changed the field of robotics. It’s the Linux for robots. When someone comes up with a new algorithm, he or she can immediately load it into ROS and make it accessible to other researchers around the world. The third element of Cloud Robotics is Open-Source: shared access to human ingenuity: code, data, and designs.

4. The fourth element of Cloud Robotics is based on robots communicating directly with each other. Consider Amazon, which has to rapidly fill thousands of orders for books and other products, packing boxes with items that can be located all over huge warehouses. A company called Kiva Systems designed a new type of robots to address this problem. The robots move around storage racks filled with boxes of items. There can be hundreds and thousands of them in a warehouse, and they increase efficiency amazingly. What makes it all work is that these robots are all talking to each other. They are constantly communicating. They work together to coordinate traffic patterns, and if conditions change, like one robot finds a bit of grease on the floor, it instantly alerts the others to avoid it. The fourth element of Cloud Robotics is Robot-to-Robot Learning: robots sharing data and code to collectively improve their performance.

5. The fifth element of cloud robotics concerns error modes, where the robot can’t determine what to do. If you work in robotics, you realize there is always going to be situations where the robot doesn’t know what to do.

When a robot gets stuck, it could ask a human for help.

The Cloud could provide access to call-centers with humans standing by to diagnose the robot’s data and video to suggest steps for recovery, when all else fails. I realize this is the opposite of what happens today, where you call technical support and a robot voice answers. One key to making this work is for the robot to monitor its confidence levels, so it can recognize when it is sufficiently uncertain and needs to stop and request help. So the fifth element of Cloud Robotics is Crowdsourcing.

Until now, Robots have been viewed as self-contained systems with limited computation and memory. Cloud Robotics suggest an exciting alternative where robots access and exchange data and code via wireless networking.

The Five Elements of Cloud Robotics are:

Cloud Robotics will build on related effort including the “Internet of Things,” “IBM’s Smarter Planet,” General Electric’s vision of the “Industrial Internet,” and Siemens’ concept of “Industry 4.0.” These approaches have enormous potential but also open a Pandora’s Box of issues related to privacy and security. But when robots have their heads in the clouds, the sky’s the limit.

Why We Love Robots is a five-minute short documentary film (co-directed with my wife, award-winning filmmaker Tiffany Shlain) that combines cultural references and found footage to explore our human fascination with robots and emerging research in Cloud Robotics. It was nominated for a 2104 Emmy Award and won a “Botscar” (Robot film Oscar) at the 2014 Robot Film Festival.

This article was originally published by The Aspen Institute on Medium