6 Reasons to Be Wary of Iowa Polls

Few things in political polling are as challenging as the Iowa caucuses.

Held every four years, the first-in-the-nation presidential voting is everything a pollster hates: complicated, unpredictable and high-profile. Nonetheless, major polling firms soldier on, in part because the interest in who might win the state is so high.

With the Iowa caucuses set to take place on Monday, pollsters large and small have taken a shot at figuring out who’s ahead. But as you read the final polls coming out of the Hawkeye State, here are six reasons to be skeptical.

The caucus process is complicated

The complicated process of assigning delegates in a caucus differs from the process of casting individual votes in a primary election, which is what polls are better equipped to measure.

“Caucusing, by it’s nature, is a little bit more—I don’t want to use the word byzantine—it’s a little more of a process,” said Jon Seaton, who ran Iowa for Republican presidential candidate John McCain in the 2008 primary. “You show up at one particular time and at one particular location, and you sit through speeches and then you cast your vote on a slip of paper.”

Patrick Murray, director of the Monmouth University Polling Institute, said precinct breakdown also complicates things.

“The votes in high-population centers are diluted because of the process,” Murray said, using the hypothetical example of 100 college students showing up to one precinct to select the same number of delegates as a dozen middle-aged farmers showing up at a rural precinct. “Polling is not able to take that into account,” Murray said.

On the Democratic side, a voter’s second-choice candidate also matters. At the precinct site, caucus-goers break into groups based on their chosen candidate. If a candidate gets less than 15% of the support at that precinct, caucus-goers are required to choose a different candidate, and Democrats from the other groups can lobby in person for their vote.

“At some point, you better be thinking pretty hard about who your second choice is, whereas you don’t have to make that consideration at all on the Republican side,” Seaton said.

Polls that don’t identify a second choice for respondents neglect to account for that dynamic.

It’s hard to tell who’s caucused previously

One of the best ways to predict the caucus outcome is to gauge the opinion of regular caucus-goers, but that information is not readily available because the caucuses are managed by the state parties.

“We don’t have good information about who’s participated in the caucus before because they’re not state-run events. They’re not part of the registered voter history,” Murray said.

The majority of registered voters in Iowa won’t caucus, and it’s difficult for pollsters to predict which among them will. When they’re called at random, Murray said people might lie about their intentions to show up. Surveying likely voters avoids that error.

“This is the first year where we have a significant number of public polls that are using voter lists to draw the samples for the polls,” Murray said. “Just dialing all possible numbers in a geographic area is overstating the number of non-voters that wind up in our sample.”

In his polls, Murray said he eliminates “low-propensity” voters, those who have not participated in the last few general elections which means about 40% of registered Iowa voters don’t even make it onto his call lists.

Turnout is a major factor, but it’s unpredictable

Murray said when he began polling for the 2016 election, he initially used a restrictive sampling frame because he anticipated the election would have particularly low turnout.

“Then it appeared that Bernie Sanders and Donald Trump were attracting a lot of new voters,” Murray said. “It wasn’t clear that they were going to show up, but it was clear that they were new.”

Murray expanded his sample to include “moderate-propensity” voters who participate in elections sporadically, the kind who might be inspired to caucus because of Trump or Sanders. In 2008, then-Sen. Barack Obama’s candidacy had the effect of turning out a lot of people who weren’t typical caucus-goers.

Curt Anderson, who was a strategist for former presidential candidate Bobby Jindal, said the big question is whether that will happen again, but he said it’s impossible to know beforehand.

“Obviously Trump is attracting some number of people who are regular caucus attendees but also additional disgruntled Republicans, so he needs higher turnout than usual. He needs people who are motivated by his message and mad at the world and who will show up at the caucus even though they normally wouldn’t,” Anderson said. “Looking at the fact that he goes places and fills arenas, I tend to think that turnout will be high.”

Polling can distract campaigns from get-out-the-vote efforts

Seaton said a more relevant predictor than polling in Iowa is which campaigns are dedicating more resources to the get-out-the-vote effort on the ground.

He said the numbers in Iowa can play tricks on a campaign, and it’s important to stay focused on get-out-the vote efforts.

“I just think it’s much more difficult to rely on survey research to craft a strategy there,” he said, adding that the challenge doesn’t exist in other states.

Iowa voters tend to be truly undecided

Murray said in most states, voters aren’t usually as undecided as they sound. By asking them a few questions, he said he can typically determine with relative certainty who they’ll vote for. But voters in Iowa and New Hampshire are different.

“I think it’s a cultural thing,” he said. “They’re just used to having all this attention thrown at them. They’re also very small states, so it’s a lot easier for voters to get out and meet them easily, so they come to expect that, and in coming to expect that, they keep their options open.”

Everything can change between the polls and the caucuses

“I think the main grain of salt is there can be swings in a primary in the days before voting,” said Sam Wang, of the Princeton Election Consortium. “So pay attention to what the No. 2 candidate is doing.”

Murray said it’s important to note that less than 60% of Iowa voters are set in their vote choice at this point, which leaves a lot of potential for movement in the final days.

It’s possible the debate performance of the Republicans candidates in the absence of Donald Trump on Thursday could sway the opinion of caucus-goers. Any polls conducted before Friday will have been unable to take that change into account.

“The voters who are showing up are paying very close attention to these things,” Murray said.

Experts tend to agree that polls are more predictive closer to the date of the caucus, but their accuracy might have a short shelf life.

“They’re accurate for today,” Murray said. “Tomorrow, those polls could be meaningless.”