Are the polls accurate this time?

Are the polls more accurate this time around?

link:https://www.nytimes.com/2020/05/12/upshot/polls-2020-trump-biden.html

Are State Polls Any Better Than They Were in 2016?

The risk of a systematic error is lower this time, but there are still issues that seem to tilt these surveys to the left.
Nate Cohn

To most, Joe Biden’s clear lead in recent state polls suggests that he has the early edge in the race for the presidency. To others, it’s not so meaningful. After all, Hillary Clinton also held a clear lead in the state polls, and yet Donald J. Trump won the election.

Four years later, it’s fair to wonder whether there is a serious risk of another systematic polling error. The answer is not cut and dried.

There is always a chance of a systematic polling error, even when the reasons aren’t evident in advance. This time, there are obvious causes for concern. Many state polls suffer from the same methodological issues that were partly or even largely responsible for the miss four years ago, despite many opportunities to improve. Yet at the same time, many of the major causes of error in 2016 seem somewhat less acute.

What’s better than in 2016: undecided voters

One major source of the 2016 polling error is much less of a factor than it was four years ago: voters who are undecided or say they will vote for a minor-party candidate. These voters appeared to break overwhelmingly toward Mr. Trump, especially in the relatively white, working-class battleground states.

This time, far fewer voters are telling pollsters they’re undecided, and that means less room for a late shift among these voters to cause a polling error. At this point in 2016, about 20 percent of voters either supported a minor-party candidate or said they were undecided. Today, the number is about half that level.

What’s a little better than in 2016: education weighting

Another source of polling error was the failure of many state pollsters to adjust their samples to adequately represent voters without a college degree. Voters with a college degree are far likelier to respond to telephone surveys than voters without one, and in 2016 the latter group was far likelier to support Mr. Trump. Over all, weighting by education shifted the typical national poll by around four percentage points toward Mr. Trump, helping explain why the national polls fared better than state polls.

Four years later, weighting by education remains just as important. The gap in the preference of white voters with or without a college degree is essentially unchanged, despite the appeal Mr. Biden was supposed to have with less educated white voters.

In the New York Times/Siena College surveys conducted in October, Mr. Biden’s combined lead over Mr. Trump in the core six battleground states — Wisconsin, Pennsylvania, Michigan, Arizona, Florida and North Carolina — was two percentage points. That lead would have been six percentage points had the polls not been weighted by education or turnout (which correlates with education).

Although they could still be doing better, more pollsters are weighting by education today than four years ago. Over all, 46 percent of the more than 30 pollsters who have released a state survey since March 1 appeared to weight by self-reported education, up from around 20 percent of battleground state pollsters in 2016.

Some of the increase is because a handful of pollsters have decided to start weighting by education, a prominent example being the Monmouth University poll. But more of the change is because of the high volume of state online polls, which have always been likelier than state telephone surveys to weight by education.

What could be worse than in 2016: new online polls

There has been a surge in new online-only state polls. Over the last month, there have been 13 such surveys, representing nearly half of the pollsters who have conducted state polls over this period. In contrast, only 10 online-only pollsters conducted surveys over the final three weeks of the 2018 election, which was about 10 percent of all of the pollsters who conducted surveys in that period.

Online polling isn’t necessarily bad. Many are sophisticated and comparable in quality to a typical live-interview telephone survey. But most of these new state polls take a simple approach: Contact the members of a large online panel, then weight those respondents by standard census demographics and maybe recalled vote choice in 2016 (more on that later). This is inexpensive and easy, but most pollsters have concluded that it’s not great. The panels just aren’t sufficiently representative, especially in small states, to expect a simple methodology to yield a high-quality result.

Until recently, few pollsters have tried to use this approach in state polling (Morning Consult is the most prolific example of a pollster that has done it nationally). But the early evidence suggests that these kind of state polls might lean to the left.

Perhaps the best early data is the AP/NORC/VoteCast polling ahead of the midterms, which combined a traditional telephone survey of 40,000 respondents with a large nonprobability online sample of 110,000 respondents. The online-only element of the survey was fairly comparable to most of the online surveys released in recent weeks, and it wouldn’t have fared well without calibration using the live-interview surveys. It would have overestimated the Democratic result by an average of about five percentage points across 71 races.

A special congressional election in West Friendship, Md., late last month. The presidential election is less than six months away.
A special congressional election in West Friendship, Md., late last month. The presidential election is less than six months away.Credit…Edwin Ramirez/EPA, via Shutterstock

Similarly, the new online polls tend to lean to the left of the state telephone polls so far this cycle. In polls conducted since March 15, Mr. Biden has run 6.6 percentage points ahead of Mrs. Clinton’s margin in online state polls, compared with a gain of 3.7 points in live-interview telephone surveys. Notably, the latter figure comes very close to Mr. Biden’s gain in national polls — about four points — over the same period.

Over the long run, these polls might make up a smaller share of the battleground polling than they have so far. But it seems inevitable that new online polls will represent a larger share of state polls this cycle, and so far the best indications are that they’ll lean to the left.

What could be worse than in 2016: recalled vote weighting

More and more, pollsters with fairly mediocre sampling methods are relying on a new tool to bring their results closer to reality: recalled 2016 vote. Here, the pollster asks respondents whether they voted for Mrs. Clinton or Mr. Trump, then adjusts the sample so that the recalled vote choice matches the actual result of the 2016 election.

Weighting on recalled vote choice certainly has its advantages. You could probably hammer even the worst survey into the ballpark. You could probably get a plausible poll result for Wyoming using a sample of New Yorkers this way.

But although this is a surefire way to reduce error, it is very hard to execute without risking a modest systematic bias. And here again, the bias would tend to be toward the Democrats.

There’s a large body of evidence suggesting that people are likelier to recall voting for the winner and less likely to recall voting for the loser. If so, polls weighting on recalled past vote would tend to be biased toward the party that lost the prior election.

It seems that this effect is, at the very least, substantially diminished compared with a decade ago. Even so, it might still be there. In the Times/Siena polls from October, for instance, 6 percent of 2016 voters refused to say whether they backed Mrs. Clinton or Mr. Trump. Those voters backed Mr. Biden by a two-to-one margin, suggesting that they were probably likelier to have supported Mrs. Clinton.

Another issue is that today’s registered voters aren’t exactly the same as those of four years ago: In the interim some people have either died, reached voting age, or moved elsewhere. So it is not appropriate to assume that people who voted in 2016, and are now registered to vote, backed Mr. Trump or Mrs. Clinton by the same margin as in the 2016 result.

It is hard to say whether this is generalizable, but over all 13 percent of the voters who took the Times/Siena polls in 2016 are no longer on their state voter file, and those voters backed Mrs. Clinton by a six-point margin, compared with a one-point lead for Mr. Trump among those who remain registered in the state. Here again, recalled vote weighting might bias a poll by only one percentage point in Mr. Biden’s favor, but the risks start to add up.

Nate Cohn is a domestic correspondent for The Upshot. He covers elections, polling and