Dems Just Keep Outperforming Expectations, and A Few More Notes on The Limits Of Polling

The Hopium Community Gathers Next Wed, Dec 13th at 7pm EST - RSVP Today

Dec 06, 2023

Today’s Toplines

More Notes on Polling, And Why I Am Optimistic About 2024 - Last week, I posted here about a series of new polls showing Biden ahead of Trump, and offered some thoughts about how to read and interpret polling these days. This post will be build on that one, so try to read that one first. The most important point - polls only tell us where things are now, and cannot tell us about where things will be tomorrow or next year. They have no predictive capacity. They are a snapshot into a moment, and that snapshot can tell us an awful lot. So polls are neither predictive, nor are they worthless as some argue. Generally more data is better than less data, and polling is best understood as a way of getting a read on where things stand with a target group of voters; but we must be mindful there are real and serious limits to what polling can and cannot tell you.

One of the areas where are there real limits in polling is in precision. Think of a poll as a sketch rather than a detailed painting. The larger the interview sample of a poll the more detailed the painting becomes. The way this is captured in data is through the “margin of error” in a published poll, something Pew Research does a very good job explaining here. All polls have built in error ranges, some far more than others. Let’s take this week’s Morning Consult tracking poll which has the race now Biden 43% and Trump 43%. It has a large number of interviews of registered voters, 5,800, which gives this poll a small margin of error, +/- 1 point. This means that Biden could at 42, 43 or 44, same with Trump. So even though the poll finds Biden-Trump tied at 43, Biden could be leading by 2, as could Trump. It’s a sketch, not a painting - there are limits to its precision.

The smaller the sample the less precision you get. So when looking at polls try to stick with those with large samples. Which brings us to the dreaded NYT polls from early October, which showed Biden “behind” in some key battleground states. The NYT did 600+ interviews in each of six states, over 3,600 in total. The state polls only had 600 interviews, which is not a lot, and this small number of interviews yielded a margin of error of between 4 and 5 points depending on the state. In Pennsylvania, for example, the margin of error was 4.6%, which means that the Trump result, 48%, could be anywhere between 43.4% and 52.6%, and the Biden result, 44%, could be anywhere between 48.6% and 39.4%. So yes, this means with this level of precision the 48%-44% Trump lead could actually be a 49%-43% Biden lead.

Are people who analyze polling overselling the accuracy and precision of polling? Yes, and it’s a problem.

The problem gets worse as the sample gets smaller. Here’s how the site Split Ticket wrote about this problem this week:

Now more than ever, political observers seem to be digging into polls’ crosstabs, oftentimes reacting with incredulity at some of the results among certain demographic subgroups. Sometimes they’ll even point to these seemingly impossible subgroup margins (e.g., “There’s just no way Trump is leading Biden among white college graduates!”) as a reason to disregard the poll’s results — and sometimes the pollster itself — entirely.
At the same time, more and more political journalists (and even some pollsters themselves) seem to be focusing their polling analysis on crosstab subgroup results almost as though they’re near-gospel.
What both of these groups fail to realize is that individual crosstabs are often very noisy. By design, breaking out survey results into subgroups results in smaller sample sizes, and thus larger margins of sampling error. For example, in the latest national Quinnipiac University poll the total sample size among registered voters is n=1,574, which would have a margin of sampling error of ±2.5%. However, if you’re just looking at results among Hispanic voters, which have a sample size of n=104, the margin of sampling error jumps to ±9.6%.
And as a reminder, margins of error apply to each number in both directions. The crosstabs from this Quinnipiac poll show Biden leading Trump 49% to 43% among Hispanic voters. With the above margin of error, Biden’s support falls within the range of 39% to 59%, while Trump’s falls within the range of 33% to 53%. These ranges are massive and overlap to such an extent that confidently drawing horse race conclusions from them becomes questionable.

Yes, even in a relatively large sample national poll of 1,500 people the number of interviews of a sub-group, in this case, Hispanics, is only 100 interviews with a 10 point margin of error. This means that Biden could be up 59%-33%, or down 53%-39%. With such a small sample the data is basically worthless. Yet, the idea that Biden is struggling with elements of his coalition - youth, Hispanics, African-Americans - has been derived largely from small sample sizes and data like this. It’s a problem.

And it’s a problem because we have not seen an erosion of our coalition in actual elections which have taken place over the past 18 months. I point this out in a new Tom Edsall NYT analysis which looks into this topic:

Simon Rosenberg, a veteran Democratic operative and former president of the New Democratic Network, emailed me a series of bullet points:

The last four presidential elections have gone 51 percent-46 percent Democratic, best run for Dems since F.D.R.’s elections. Only 1 R — George W. Bush 2004 — has broken 48 percent since the 1992 election, and Dems have won more votes in seven of last eight presidential elections. If there is a party with a coalition problem, it is them, not us.
Our performance since Dobbs remains remarkable, and important. In 2022 we gained in AZ, CO, GA, MI, MN, NH, PA over 2020, getting to 59 percent in CO, 57 percent in PA, 55 percent in MI, 54 percent in NH in that “red wave” year. This year we’ve won and outperformed across the country in every kind of election, essentially leaving this a blue wave year.
We got to 56 percent in the WI SCOTUS race, 57 percent in Ohio, flipped Colorado Springs and Jacksonville, flipped the VA House, Kentucky Governor Andrew Beshear grew his margin, we won mayoralties and school board races across the United States. Elections are about winning and losing, and we keep winning and they keep losing.

In a recent post on his Substack, “Why I Am Optimistic About 2024,” Rosenberg elaborated:
“Opposition and fear of MAGA is the dominant force in U.S. politics today, and that is a big problem for super-MAGA Trump in 2024. Fear and opposition to MAGA has been propelling our electoral wins since 2018, and will almost certainly do so again next year.”

So, Simon, are there large sample polls of the sub-groups, and what do they tell us?

We do have recent large sample polls of Hispanics and young people, and both show Biden at, near or above his 2020 numbers:

Hispanics - A late September large sample (1401 interviews) bi-partisan Univision poll just of Hispanic voters had Biden leading Trump 58%-31% (+27). This would put Biden at or above of two of the three Hispanic exit polls from 2020, and within striking range of what I think his number needs to be in 2024, +30. Remember the last two cycles have been the best Democrats have had in the heavily Hispanic Southwest since the 1940s (yes there’s that winning elections thing).

Do note that the NYT poll which caused the big freak-out had Biden +8 with Hispanics in these battleground states. Here it’s +27.

Young People - A new gold standard Harvard/IOP youth poll (2098 interviews) has Biden up over Trump with likely 18-29 year old voters 57%-33% (+24). +24 was Biden’s margin in 2020 according to the Exit polls. His lead shrinks to +16 with likely young voters when all the potential third party candidates are named so while this poll is encouraging we still have work to do.

In what is very bad news for Rs, despite having a live primary right now, there’s been a significant drop in vote intent for young Republicans from four years ago while Dems are holding:

Democrats (Fall 2019: 68% “definitely vote,” Fall 2023: 66%)
Republicans (Fall 2019: 66%, Fall 2023: 56%)
Independent/Unaffiliated (Fall 2019: 41%, Fall 2023: 31%)

The NYT battleground poll had Biden winning 18-29 year olds by only a single point, and losing 18-44 year old voters overall. Here Biden is winning 18-29 year olds by 24 points, and 2020 exits had Biden winning 18-44 year olds by 14 pts in 2020.

Yes these two large sample polls are a bit of a narrative buster for the frayed Dem coalition argument.

In her latest Washington Post column Jennifer Rubin also cites our recent analysis, and reminds us that another large sample youth poll conducted by CIRCLE at Tisch College at Tufts University also finds Democrats doing well with young people:

My dim view of polling a year out from the election is no secret. To illustrate the foolishness of building punditry around meaningless, premature polling, consider what would unfold if pundits ran with a spate of recent polling in President Biden’s favor.
Democratic consultant Simon Rosenberg pointed out on his website that four recent polls show Biden with small leads over four-time indicted former president Donald Trump:

44 percent-42 percent Economist/YouGov; Trump led 43 percent-42 percent in mid-November
43 percent-42 percent Morning Consult; Trump led 44 percent-41 percent the prior week
39 percent-37 percent YouGov (this is a separate poll by the same firm)
37 percent-35 percent Leger/The Canadian Press

Rosenberg pointed out that in the Morning Consult poll, Biden gained four percentage points over the prior week, and he gained three percentage points in the Economist/YouGov poll.
Consider also a recent, mammoth survey of young voters. This comes amid a raft of “analysis” declaring Biden had lost young voters, in part because of his handling of the Israel-Gaza war. “Among youth who say they’re extremely likely to vote, 51% prefer a Democratic candidate in 2024, 30% a Republican, and 16% are undecided,” the Center for Information & Research on Civic Learning and Engagement at Tufts University found in a poll of 2,017 Americans ages 18 to 34. “Among all youth, regardless of likelihood to vote, 37% say they’ll vote for a Democrat, 25% a Republican, and nearly a third (31%) say they don’t yet know who they’ll support.” By comparison, Biden won in 2020 voters younger than 30 by 24 points and those aged 30 through 44 by only six points…..

By the way, here’s an update on the recent national polling Jen Rubin sites. We now have six now with Biden tied or ahead in the last week (via 538):

42-41 Economist/YouGov (Biden’s gained 2 points since mid-November)
43-43 Morning Consult (Biden’s gained 3 points since late November)
39-37 YouGov
37-35 Leger/The Canadian Press
44-42 Economist/You Gov (last week)
43-42 Morning Consult (last week)

Putting this all together, some thoughts:

Biden either leads or is tied in the Presidential election in post-Thanksgiving polls - Polls which assert he is behind should no longer be used in analysis. The Edsall article above for example, should not have cited a poll taken in early November. It no longer speaks to the political moment, which is more favorable to Biden. Similarly, the NYT polls from October should no longer be referenced. The only fair characterization of the election now is that polls show it within margin of error, with a majority of them showing Biden with a slight lead. Trump is no longer favored or leading in the polls.

Recent large sample polls of Hispanics and young people do not find erosion for Biden and the Democrats - Analysts have to be far more careful about making assertions about the state of sub-groups based on very small sub-group samples in broader polling. If either of our two parties is having coalition problems right now, it is clearly the one which keeps losing elections across the country and which has lost the last 4 Presidential elections by an average of 5 points 51%-46%; whose nominee is at 60% in the primary and not in the upper 70s; and which has a dedicated, effective and very prominent group of former members who have splintered off working against it everyday.

Additionally polls which have sub-group results which seem way out of wack with other published polling, or recent election performance (like the NYT polls), need to do a lot of thinking about running with their results, or make clear how much of an outlier they are. Democrats have been +10 or more with 18-44 year olds in every election since 2016; the idea that all of a sudden Trump jumped out ahead of them with these voters, as the NYT polls found, is implausible, particularly given our heightened electorate performance this year. There is just no actual example on the ground of the Democratic coalition fraying this year, or in the battlegrounds last year.

Our continued strength in elections of all kinds all across the country since the spring of 2022 is in my mind the most important electoral data out there, and should be getting far more attention - it would be wise for the political commentary community to be a bit more honest about the limitations of polling, and work harder to bring in other data - electoral performance, fundraising, early vote, voter registration data - to help provide a more detailed and complete picture of our politics right now. Bringing in this other data was a central reason Tom Bonier and I got 2022 right when so many got it wrong, and why my commentary this year has been far more accurate than most. Polling is incredibly valuable, but it has limits, and I’ve have worked hard to bring in other data to provide a much more complete picture for all of you.

Note that last night Democrats had another significant overperformance in a South Florida state house special election - 10 pts above 2020, 31 pts over 2022. Amazing stuff:

There are too many Republican-allied polls and pollsters in the averages - This is something Tom Bonier and I discussed a lot last year, but the percentage of polls that feed into our understanding of what’s happening coming from Republican-allied pollsters or organizations is just too high right now. I will be coming back to this in a future post. It’s a major ongoing data integrity problem.

The bottom line - Joe Biden is a good President, the country is better off, the Democratic Party is strong and keeps winning elections across the country. I am comfortable with where are now, am very optimistic about 2024, and look forward to kicking ass and gettin’ it done with all of you next year.

Keep working hard all - Simon

Hopium Chronicles By Simon Rosenberg

Discussion about this post