Van Gogh and the Algorithm: How Math Can Save Art

It took X-ray fluorescence spectroscopy and a painting algorithm to reveal the hidden portrait of a peasant underneath the painting of Vincent Van Gogh’s “Patch of Grass.” And that feat, accomplished in 2008, was just the beginning.

Art history and mathematics may seem an unlikely combination, but math techniques in image analysis is transforming the way art historians and conservationists do their work. Algorithms can be used to distinguish copies from originals, characterize and quantify the style of one artist in comparison to another, and restore cracks and fading. “What’s really important to note is that these are all non-invasive techniques,” says William Brown, chief conservator of the North Carolina Museum of Art, who gathered last week in Washington to discuss his work at a panel discussion sponsored by Duke University.

While many restoration techniques interfere with art’s chemical composition, the image analysis technique leaves art untouched. Signal processing extracts X-ray images of the art, allowing it to be viewed in enhanced or altered forms while the original remains.

“Basically, there is a mixture of sources in the artwork and we want to make sense of the mix and which elements are more present than others,” says Robert Calderbank, director of the Information Initiative at Duke. Calderbank likened the technique to the study of skin cancer, in which scientists separate different types of melatonin from within the same skin lesion.

Brown and Ingrid Daubechies, professor of mathematics at Duke, having been experimenting with the technique since 2011. The two first collaborated to characterize the style of Giotto di Bondone’s Peruzzi Altarpiece. For generations, scholars had noted that some of the faces looked much more naturalistic than others, suggesting the possibility that different artists, perhaps di Bondone’s apprentices, had painted portions of the altarpiece. By characterizing combinations, such as detail elements of brushstrokes, and using charts to visualize information and patterns, Brown and Daubechies were able to determine which panels were the ones likely painted by di Bondone, and which by his apprentices.

A restoration of another altarpiece, the Ghent Altarpiece, made it possible to distinguish once illegibly cracked calligraphy as the text of Thomas Aquinas on the Annunciation. Using an image-processing algorithm, Brown virtually removed the distortion created by the painting’s cradle resulting in an X-ray image of the art, sans cradle modification.

Daubechies is confident that the interdisciplinary work, enabled by data, can produce what neither mathematics nor art could produce independently. “We want to give you a third eye,” Daubechies said of entry of mathematics into the art world. “It’s not competitive.”