Some Notes on Polling, 18 Months Out

No Freaking Out About A Single Poll Please

Many of you have asked about the Washington Post/ABC News poll which came out on Sunday, and so I’ve put together some notes about how we can start thinking about polling and electoral analysis this cycle here at Hopium. Let me know if you find all this helpful.

Tom Bonier and I are working on a longer, more in-depth piece so today will be something more like some loose notes than a big comprehensive memo……

Polls Don’t Predict, They Are A Snapshot in Time - Every poll matters and every poll can tell you something. What polls cannot do is predict the future. All they can do is tell you what is happening today. That’s why polls like Navigator which track things over time are so important. They can compare today to a few weeks ago, last year, etc and it is that comparison of data over time that provides the best understandings of politics.

Also, think of polls as sketches, not a full painting. They give you a rough guide to where you are, but are not 100% accurate and never will be. So yes, all poll analysis has to be done with a great deal of humility, as there is significant uncertainty built into the process.

Stick to Averages and Trends, Never Allow A Single Poll To Dictate Your Understanding - Just as we should look at data over time, we should look at data over as many polls as we can. Polls can get things wrong, are rough as discussed above, which is why the more polls and data you have the more accurate your insights are likely to be.

Despite its flaws, 538 is still the best poll aggregator out there. I check it first thing every morning. Here’s a link to its Biden approval rating. Note how wide the variance is - mid 30s up to the low 50s - with an average of 43%. 43% is about where the President was in the fall of last year when Democrats racked up big wins in many of the battlegrounds which will determine the Presidential election next year. So while it isn’t high, it’s a number Democrats can live with right now.

Real Clear Politics while once a serious site has become a right wing propaganda outfit and should be ignored, as should any analyst referring to its data and insights.

Polling Has Gotten Expensive and Hard - Polling is hard. We are a remarkably diverse country, and getting samples which accurately capture a state or district is really hard. It has gotten harder because response rates to traditional polling have declined, making it more expensive to get a good and accurate sample. That extra cost and difficulty in constructing a good sample can encourage pollsters to cut corners a bit leading to lower-quality data.

The big NYTimes story on the red wave that never came does a good job of explaining this dynamic.

The Amount of Public Polling Being Done By Right Leaning Outfits Is Increasing - The recent proliferation of right-leaning public polls was at the core of my battle against the red wave in the final weeks of the election. It used to be that a majority of polling was done by independent pollsters, often media organizations or university projects. Last year we saw a significant increase in right-leaning partisan public polling. Historically partisan polling remains private, as candidates and party committees do not want the other side to see what they are seeing. But in the last year there has been an increase in public, non-candidate/non-party polls by right-leaning outfits. In the 538 Biden approval average, I linked to earlier, there are roughly as many right-leaning polls as independent media polls and just a few left-leaning outfits. This means right-leaning money is playing a much bigger role today in shaping our understanding of polls and politics (yes the right is louder than we are), which is something we have to discuss in the coming months. It’s not sustainable, for Democrats cannot be expected to take poll averages seriously where 40-50% of the polling is being done by outfits on the right.

For it was a flood of these right-leaning polls that threw off the averages late in the last election causing almost everyone to get the election wrong (below is the infamous Axios “Red Tsunami” graphic from 10/23/22).

What About that WaPo/ABC Poll? Okay, that takes us to the Washington Post/ABC News poll that came out Sunday. Simply, it wasn’t a good poll and probably shouldn’t have been published. First, it polled adults, not just registered voters. Given all the problems we’ve had with polling and accuracy in recent years, it’s my opinion that pollsters should only be polling registered voters. What value is there in asking someone who cannot vote who they are voting for? Polling adults saves money but leads to junkier data.

Next, and this is something both Tom Bonier and our friend Stephen Clermont at Change Research have very strong feelings about, some of the demographic breakouts of the top-line data are just hard to believe and raise serious questions about the integrity of the poll itself. The most obvious example is that the poll has Trump winning younger voters and Biden winning older voters. Based on the election results of the last 3 elections, and lots of other polling this year, we know that these findings cannot be true. These kinds of things happen in polling, but when they do pollsters should either 1) acknowledge the problems 2) not publish the findings. Given these problems and others in the poll, it is my view this poll should have not been published, particularly by such reputable news organizations (I once worked at ABC News and have very high regard for the news division).

So, a reminder - stick to the averages (and yes they have too many R polls right now), watch trends over time, don’t freak out over a single poll either way, be patient and measured.

I haven’t really geared up my own electoral analysis machine yet because it is early, and we don’t yet know who will be running and what kind of shape they will be in next spring when it all really starts to matter. I don’t plan on writing about individual polls but will be writing about broader trends at this point - things like how unpopular the Republicans are, Biden’s underperformance on economic issues, etc. We’ve already had a fair amount of that so far here at Hopium.

I will also be incorporating into my analysis this cycle the most important parts of the analysis Tom and I did last cycle, which looks at data outside of polling - voter registration figures, fundraising, election results in places like Wisconsin last month or Jacksonville next week, early vote performance - that help paint a clearer and more detailed picture than what polling alone can provide. As I have already discussed with many reporters this cycle, I would be very careful about drawing conclusions about the intensity of the Democratic family from current polling. It didn’t capture the intensity we saw in the battlegrounds last year, and we already saw a fired-up Democratic machine blow it out in Wisconsin last month. I have no doubt that the anti-MAGA majority which has showed up in such big numbers in the last three elections is going to do so again next year with either Trump or DeSantis (two sides of the MAGA coin) on the ballot next year.

The central reason so many analysts got the election so wrong last time is they underestimated how powerful a force the fear of MAGA is in our politics, and how mature and muscular our grassroots have become. Analysts underestimated the intensity of Democrats last cycle and should be very very careful about doing so this time too.

For those who want to read a bit more about how Tom and I got it right when so many got it so wrong last time, here are a few things to check out:

My Post-Election Memo - The Democratic Party Is Strong, the Rs Remain All MAGA

Still Waiting On The Red Wave - A Recap of Our Heralded 2022 Election Analysis

Our Most Comprehensive Public Discussion Of What We Did (Video)

The ‘Red Wave’ Washout: How Skewed Polls Fed A False Election Narrative

My Interview With Ron Brownstein in The Atlantic

Keep working hard all - Simon

P.S. A new article on the 2024 Presidential in The Hill elections has these related passages:

Another prominent Democrat, veteran political strategist Simon Rosenberg, who predicted his party’s success in the midterms against the conventional polling wisdom, said there are more indicators now that point to Democrats’ momentum.  

The party gets better organized each cycle and some of the unexpected wins in 2022 were built on the groundwork laid in 2018 and 2020, he said. More engagement, grassroots volunteers, and small-dollar donations are pouring in to help fuel the party in a way he argues the GOP is underestimating.  

“Commentators have to be very cautious about gauging intensity levels in this next election,” Rosenberg said.  

The roiling politics of abortion are another factor, something seen in Wisconsin, where Biden narrowly defeated Trump in the last election. After the Supreme Court’s decision to overturn Roe v. Wade, voters came out in support of the Democratic candidate for state Supreme Court.  

“You’ve now had three consecutive elections where Democrats’ performance has been at the upper end of what’s been possible for us,” Rosenberg said.