Are the Polls Trash? Maybe. Are You Over-Reading Them? Definitely

This article is part of The D.C. Brief, TIME’s politics newsletter. Sign up here to get stories like this sent to your inbox.

For most of any two-year election cycle, but especially as we get closer to Election Day, we are flying blind. Despite all the campaign ads and rehearsed speeches and faux scandals and, in some rare cases, actual in-person debates, we don’t know which team is going to do better turning their side out. We don’t like sitting with that uncertainty, so we turn to the political junkie’s drug of choice: polls.

Voting began way back in the tail end of summer, when the most aggressive early-voting states of Minnesota and South Dakota started casting ballots in September. The first batch of real results don’t come until after polls close on Election Day, and at that point, they come with such ferocity that it’s easy to lose track of surprise wins—and losses—as all 435 House seats, roughly a third of the Senate, and three dozen governorships are decided.

In lieu, those of us who make careers in the political ecosystem substitute proxies: campaign finance reports, TV ad reservations, digital and social outlays, candidate visits, voter-registration data, even consultant gossip about who’s in and who’s out. But we lean more on polling than any of those other vices. And the problem there is that what should be just another informative input has become treated as predictive.

“You’re reading these as tea leaves. They’re hieroglyphics,” one pollster texted me back yesterday. “If you don’t know how to read them, you’re doing major harm.”

That’s not an uncommon grievance among polling pros. And for good reason. And I am as guilty as anyone.

Public-opinion surveys have become a substitute for actual analysis. Take, for instance, the shift in a recent New York Times/ Siena poll’s most-predictive question. When asked who should control Congress next year, Republicans appeared to have a 4-point advantage over Democrats; read the actual numbers, it’s closer to 2.5 points. Negligible? Sure. But not entirely a rounding error.

Go a little down-stream in the polling numbers, and it gets far dicier. The Times’ September poll showed independent women favoring Democrats by a fulsome 14 points. A month later, they’re backing Republicans by 18 points. That’s an epic 32-point swing, one that is almost impossible to imagine in practice. But then you look at the details; the embedded margin of error for the small sample size is 20 points. Put simply: they didn’t talk to enough of those independent women to know what they’re talking about, but the rest of us can’t shut up about it.

And this is the problem with polling right now: we treat it as Gospel. Polls have consistently—with the exception of 2012—given Democrats too much of a leg up in recent cycles. Pollsters have missed the turn-out universe by factors of two to eight points. The Trump era contributed to a surge in voters refusing to engage with pollsters, resulting in bogus numbers. The pros are already anticipating garbage data in the stream. And fickle analysis has left confidence in the polls shaky at best and ruinous in reality.

Still, most of the armchair political analysts look at the numbers as though they hold answers. Frank talk: they do not. They inform part of the discussion. They spot which campaigns might be toast. The polls can help decide whether a campaign should follow Twitter’s advice and make social justice the hill to die on or to stick with the economy. Kitchen-table issues like inflation and gas prices have dinged Democrats plenty hard, but it’s also been a half century since abortion was as in play as it is now. Polling provides some hints as to what is resonating and what isn’t, but no politician who wants to be seen as a leader defaults to what a pollster tells them.

In this, Speaker Nancy Pelosi wasn’t wrong when, in recent weeks, she began telling allies that every single race is its own universe with its own peculiarities. National headwinds don’t necessarily blow so hard in the Richmond suburbs, for instance. If the GOP holds every seat that was in Trump’s column in 2020 and carries all of the Biden districts that Biden won by five points or less, Republicans will still only have a six-seat majority—hardly a landslide.

While there’s plenty—emphasis: plenty—of reason to roll eyes at the assertion that Democrats actually could gain seats this cycle, there’s at least a rational argument that things might not be turn out as dire as national forecasts anticipate. Anecdotally, things don’t feel so despondent out there for liberals, but there’s no real evidence to back that up.

At least no evidence beyond the notions fed by polling.

Make sense of what matters in Washington. Sign up for the D.C. Brief newsletter.