Inside the Studio With an AI-Guided Painting Robot

To help illustrate our cover story on how the AI arms race is changing the world, we reached out to award-winning AI artist Pindar Van Arman, who uses artificial intelligence to create his art.

Van Arman, who built his first “painting robot” 15 years ago, uses deep learning neural networks, artificial intelligence, feedback loops, and computational creativity to guide his newer robots. As a result, the robots end up making a surprising number of independent aesthetic decisions in the course of painting each piece—putting a different spin on the idea of “generative” AI: artificial intelligence that doesn’t just compute, but also creates.

“My machines have grown beyond being simple assistants and are now effectively augmenting my own creativity, as well as having creativity of their own,” says Van Arman. “They have become a generative AI art system so sophisticated that it has forced me to consider the possibility that all art is generative.”

While generative AI drawing systems like Dall-E and Midjourney have recently become popular by creating images from text prompts in a matter of seconds, Van Arman’s 6-ft.-by-4-ft. robot spent 22 hours and 13,940 brush strokes to create a self-portrait of itself and the artist.

Read the cover story: The AI Arms Race Is Changing Everything

Here’s how Van Arman describes the process:

First, the background was created by starting from a random arrangement of 1s and 0s—not simply arranged by a conventional random-number generator, which is not random enough for Van Arman’s purposes, but instead generated by measuring atoms in superposition using one of IBM’s quantum computers. “At the atomic level is the only place we know of where true randomness occurs,” he says. The results were then printed in a very small pattern on the back of the canvas.

“We then took a portrait photo of me, and a photo of the robot. The qubits from the previous step were used to introduce errors and scramble the photos of me and the robot,” Van Arman says, “and 500 scrambled images were made of each, with various amounts of distortion.”

The 500 images were then fed into a generative adversarial network (GAN)—a type of AI framework—that looked for patterns and essentially unscrambled the images. “You can see with the robot, it did well, but with my face, the GANs were still confused, and thus the double image.”

Van Arman then combined his portrait with the robot’s portrait and gave the result to the robot, which was then given eight translucent paints and told to make the canvas look like the image from the unscrambled photos. The background of the piece was prompted using AI to imagine deep space.

“It would then make that brushstroke and a couple others, step back, look at the canvas, and ask itself again, ‘Where can I apply a brushstroke to make the canvas look more like the image in my memory?’—over and over again until the images emerged,” explains Van Arman. “I changed brush strokes from wide to more fine as the process progressed.”

So, after all that, can a machine actually be creative?

Van Arman is convinced. “Yes: anytime someone or something solves a problem without using a brute force solution, they are being creative,” he says. “AI and even simpler software programs do this all the time.”