'Smaller data can come to conclusions that bigger data can’t'
On Nov. 2, 2016, Nate Silver, Sam Wang, Nate Cohn and other major poll forecasters put Hillary Clinton’s chances of victory in the 70% to 99% range. The Election Day outcome shows how risky, myopic and even dangerous big data can be. At the same time, it’s a valuable reminder that smaller data can come to conclusions that bigger data can’t.
While visiting middle-income consumers in nearly a dozen U.S. states in an effort to understand the current market and mindset for three international brands eager to break into the American market, it became clear that most voters balance both a public and a private persona. In contrast to the scrappy, elbowing, media-fed image of the average Trump supporter, no one was deplorable. There were few if any Trump signs dotting any lawns. On the surface, the men and women who later declared their affiliation to Trump could have thrown their support behind either candidate. Still, even in homes with a Hillary sign on the lawn, it soon became obvious to me and my research team which one of the two candidates the resident was voting for, and it wasn’t the Democrat. All a visitor had to do was observe, ask questions, and listen to what was left unsaid.
First, the more concrete, spoken part. As both men and women talked to my five-member team about a variety of non-political topics, very often words like “values” would come up, along with “community” and “sticking together.” Many seemed nostalgic for more straightforward times. Often they had on display a collection of some kind—figurines, or a shelf of photographs showing a wedding, a child, a memorable vacation. Many of these photos reflected a simpler, more naive and yes, more homogenously white America. Nine times of 10, these people later confessed to my team and me that they were voting for Trump.
Also telling, in the run-up to Halloween, was the popularity of Trump masks and costumes. According to two companies, HalloweenCostumes.com, and SpiritHalloween, Trump outfits and masks outsold Hillary-wear in 2016 by 10-to-15 percentage points. Many parents who accompanied their young kids around the neighborhood this year went as Trump. There’s a German word, Maskenfreiheit, meaning “the freedom conferred by masks.” But instead of mocking or caricaturing the candidate, it seemed that slipping on a latex Trump mask made wearers believe they were stripping off their own gags.
Also, in roughly 65% of the 121 homes we visited in 11 U.S. states were the remnants of so-called man-caves, those leathery, dark-walled, tech-heavy basements and cellars where American males are said to feel free to be themselves. Again with no knowledge of the residents’ political affiliations, in houses where a man-cave had been repurposed into, say, a kids’ playroom, leaving ghostly outlines on the wall where guns, tools or sports posters once hung, my team later deduced that this was the home of a Trump supporter, even in residences where a Hillary/Kaine sign stood on the front lawn. “My daughter put that there,” explained one man who later said he was voting for Trump, adding, “She’s into that stuff.”
Still, the biggest indicators that the residents supported the Republican candidate were—of all things—silence, politeness and civility. In the course of a campaign in which Trump took direct, cruelly zestful aim at American political correctness, it was the absence of strong opinions, and the avoidance of most controversial subjects, that characterized people who later confessed their support for Trump. If more than three-quarters of our representative sample of Clinton supporters, split across multiple factors including religion, income, demographics and key interests, were frank, outspoken and collusive in their opinions, an equally sizable majority of Trump voters kept their true thoughts mostly to themselves.
In the end, my team and I concluded that many Trump supporters could be characterized by the very same political correctness their candidate railed against, that theirs was the affable, simmering quiet of a religious congregation focused on the political equivalent of Revelations. (One middle-aged female Trump voter even told me that her support gave her a kind of “gaydar” that allowed her to intuit when a fellow Trump supporter was in the room.) In striking contrast to Clinton supporters, there seemed to be a collective agreement among Trump supporters to smile, keep quiet and change the subject—either because it was too socially reckless to show off their allegiance, or they didn’t want to associate themselves with the fringe element the media kept on insisting they were.
How can all this seemingly random small data come together to predict a national election? Answer: the Trump supporters in 10 American states bore almost no resemblance to their media-made image. On the outside, they lived in homes whose residents could have easily cast their vote for either of the two candidates. But inside—in Halloween masks, photographic displays, vacated man-caves and, most of all, in the uniform political correctness and conversational caution their own candidate took pains to deride—lay clues to a movement that deftly misled one pollster after another. Trumpism was a religion, and talking about your religion openly is impolite, even taboo, in America.
The bigger point is that big data, and predictive analytics, has become a straightjacket, one that’s been marketed as a definitive solution to who and what we are. It paralyzes us to see one thought pattern while bypassing hearts, minds and everything else that lies in-between. Rather than relying on terabytes, pollsters and analysts need to spend time with people in their own homes, asking questions and getting responses, or none at all, that sometimes speak more loudly than words.
TIME Ideas hosts the world's leading voices, providing commentary on events in news, society, and culture. We welcome outside contributions. Opinions expressed do not necessarily reflect the views of TIME editors.