Is there some analysis why the polls didn't correctly predict the result?
A failure in representative polls like this should be avoided with statistical methods.
A failure in representative polls like this should be avoided with statistical methods.
But I don't know how big a factor this is in reality versus the economy.
In 2016, the majority of outlets gave Clinton a 90% chance or more. This time almost everyone said it was 50:50. The result is somewhat similar, the predictions could hardly be more different.
For one, they said Clinton had a 70% chance of winning.
But perhaps more importantly, people's poor understanding of stats meant that many people interpreted that as "She's going to get 70% of the vote" (i.e., a landslide, "and so I don't need to vote").
No, they didn't.
https://www.nytimes.com/interactive/2016/upshot/presidential...
https://www.reuters.com/article/world/clinton-has-90-percent...
Democrats were absolutely demonized by Trump, but their trust in the media is double that of Independents and quadruple that of Republicans. So to the extent that pollsters are treated as part of the media, they'll get more accurate answers out of Democrats.
[0] https://news.gallup.com/poll/651977/americans-trust-media-re...
You can try an experiment where you say out loud that you vote for Trump in your work place, and you can see how it goes.
People support Democrats visibly at their workplaces, and that is totally fine. But if you support Republicans visibly, you will have a lot of issues at work.
So, if a poller asks you who you would vote for, you would not risk it. You would just say Dems or just avoid the poll altogether.
It's a weird spot, because most Trump supporters don't really support him. They consider him a liar at worse, and a jokester at best. Telling Trump supporters what Trump has said is almost always met with "well he didn't mean that!"
Why do people vote for someone they don't even believe? Not sure, maybe they hate the other side just that much, or maybe they do believe him but don't want to admit it. Maybe they're hoping deep down Trump really is just joking.
Thank you for adding to the point that you can't let your colleagues know that you voted for Trump. Your work life would be completely fucked.
You are an ideal example. You think Trump supporters are evil. And how do you handle evil people? You make sure they don't succeed in life.
Not mine, but actually every single company in the US, yours included. I don't know of any companies that tolerate racism or bigotry in general.
I don't think Trump supporters or conservatives are evil at all. I think they're largely radicalized at this point. Most conservatives I meet seem completely unable to voice conservative opinions without resorting to bigotry - this includes our president elect.
There's nothing wrong with conservative opinions and actually everyone is on the same page about that. But can you be pro-life without calling women sluts and whores? Can you be anti-gay without telling people they deserve AIDS? Can you be anti-trans without calling them tra*ies?
For a lot of conservatives, the answer to those questions is no. Okay, then the next question is - is that type of behavior allowed at work? Of course not.
So put the pieces together yourself - where does that leave those people? This isn't a rhetorical question.
I don't know whether a poll is anonymous. The poll has no impact on my life. I'll either lie to be safe or just don't take the poll at all.
The margin in Pennsylvania will continue to shrink, as the only place with lots of votes left to count is Philadelphia. Michigan might still flip blue, because the only place with votes to count is Detroit. Arizona is still a total coin toss, with 51k vote difference and >1200k votes left to count. Wisconsin is going to be close too, although it will likely stay red.
None of that matters when there are less ballots left to count than the margin in PA, but still, the message from the polls before election was "this will be a nailbiter", and it kind of was.
What they said was that they could not predict the outcome, and were giving basically 50/50 odds of either candidate winning, which is essentially just another way of saying "I have no idea".
Just because their odds were 50/50 though, does not mean the outcome would be close. The pollsters were all warning that the swing states would likely be strongly correlated, so if a candidate performed strongly in one swing state, they'd probably perform strongly in all of them.
Nate Silver has recently written about the clear problems in polling, and in particular the herd-like way they were reporting implausible numbers:
https://www.natesilver.net/p/theres-more-herding-in-swing-st...
the most likely result predicted by 538 was 312 for trump [0]
the issue with the model was the 2nd most likely result was 319 for harris.
they thought the odds of a recount being decisive was around 10%.
That hardly seems evidence of "predicting a split election". which prediction are you thinking of?
[0] https://projects.fivethirtyeight.com/2024-election-forecast/
I don't know what Nate Silver was predicting. Was he predicting a near-split election or the situation where "someone is decently likely to win decisively, but we don't know who"?
Who was? A 50% chance to win does not imply that the vote count will be close.
Also: statistical uncertainty is a feature not a bug. A lot of the idea behind statistics is the ability to quantify the certainty of the point estimate. As another commenter put it: a statistically sound "idk" is a better result than a confidently incorrect estimate, from a statistical standpoint.
The data he has to back up his "too close results to be true random polls" is fantastic.
Although I don’t actually think it was equally likely like that, we are missing something to make all this analysis actually informative rather than a “all I know is that I don’t know anything”. We had mountains of evidence indicating that it was totally unclear, so frustrating. Perhaps that’s how the probabilities actually were, but somehow guts pointed to Trump much more regardless of personal bias, and in hindsight it feels rather obvious. Confirmation bias I guess, I still want trust all the expert analysis.
It's just incredibly hard to build a representative sample of the population.
It used to be "Libertarian" which for a subset was "I'm a Republican who likes to smoke weed".
Source?
The polls were better but still consistently underestimated Trumps support by a lot. Basically, the weighting they do for the polls now basically just guarantees that they converge on the results of the last election.
The point is no poll caught any of the swings at all. To win with this margin Trump the polls can hardly be tied and be called accurate.
The result is not a close at all, and it is not about swing states and electoral college swings. Trump is winning the popular vote by a large margin something he has never be able to do so before.
It is not like no trump supporter participated in the polls and he got 10% in them, they all showed 45-47% of their samples consistently supporting him. Those 45% apparently didn't think they were marginalized by media enough no to respond and express their support, which is most of them, so I don't see this is the widely held sentiment
What you are basically saying is exact same percentage of people who favor trump over Harris is the same % percentage of people who don't care to respond to polls and cancel each other out, that is extremely unlikely.
Also know that bias modelled in any poll already, not just response rate bias for trump voters ,all kinds of selection biases(likely to respond) for different demographics get factored in typically with past data
This is outside of the margin of error.
All of the polls had Trump ahead there. https://projects.fivethirtyeight.com/polls/president-general... (No intended endorsement of 538, they're just a convenient list of polls)
So, not really even within the margin of error. There weren’t any predicting +15.
And the high quality polls that didn’t attempt to synthesize results to match past results were even more wrong.
even nate silver called then the most accurate pollster during the 2020 race.
My explanation for this is that most polls were fabricated, showing enthusiasm for (D) which wasn't there. Basically, a form of propaganda. The most striking example here is Selzer, with that Harris+3 Iowa poll the day before the election.
It feels like there has to be a better way to present the data to make it more obvious what's actually happening.
In reality, a lot more people have traditional values when it comes to race, LGBT whatever, sexism, spiritual values, opinions on Russia, Israel etc. However in public they may be scared to voice their true opinions.
If I predict a coin toss to be 50/50 that doesn’t mean I expect it to land on its side.
There was no 50/50 chance of the voter base waking up and instead voting for Kamala yesterday.
From a frequentist perspective, it makes no sense to talk about probabilities of the outcomes of processes that can't be repeated, such as elections. So the question is then, "Why couldn't the polls predict a result?" And we know the answer: because the polls weren't precise enough. We already knew that.
From a Bayesian perspective, lack of knowledge is the same thing as nondeterminism in the underlying processes. So, to a Bayesian, you're just wrong; there was a 52/48 chance of the voter base waking up and instead voting for Kamala yesterday.
If from some other formulation, which?
This is so confused. The probability models are designed to describe situations where cause and effect is not known.causes still exist whether you can repeat them in an experiment,
You are confusing logical models with real world decisions and actions.
There is no sense in which Harris had a 50% chance and had an unlucky day. The only “chance” going on is how likely the poll sample represents the population. The math behind that assumes you have a genuine sample and ignores realities like preference falsification.
Please think and read charitably before making personal attacks. I generally take that as a sign you are acting in bad faith and I do not want to interact with you. Goodbye.
Use Polymarket instead, where money is on the line.
I don't disagree that the one french dude betting 30M on Trump on polymarket showed that there isn't enough liquidity in such markets for such distortions to be corrected, but whalebait on Manifold is not really related.
That’s not true when it comes to things like bigotry.
The people in a swing state choosing to spend time responding to polls are insufficiently representative. They're drowning in advertisements, calls, texts, unexpected people at their door and randos on the street. Why would they give time to a pollster?
For example, from Nate Silver's blog:
> The Silver Bulletin polling averages are a little fancy. They adjust for whether polls are conducted among registered or likely voters and house effects. They weight more reliable polls more heavily. And they use national polls to make inferences about state polls and vice versa. It requires a few extra CPU cycles — but the reward is a more stable average that doesn’t get psyched out by outliers.
All this weighting and massaging and inferencing results in results that are basically wrong.
Come Election Night he basically threw the whole thing in the trash too!
It is like when your doctor is asking you if you eat fast food — some people will downplay it because they know it is wrong, but do it anyways in a "weak" moment when nobody is looking.
So suddenly in my village where I know everybody 56% voted for the right wing candidate, yet everybody¹ claimed not to do that when asked before or after.
¹: except one or two open Nazis
After he was finally disgraced fully enough to resign with some remaining dignity, you couldn't find anybody who admitted to voting for him.
And he had been re-elected to a second term !
- Who does or does not have a phone.
- Who will or won't answer a phone.
- Specific target demographics, under the rubric of "stratified random sampling".
Just a few off the top of my head.
As I'd mentioned upstream, political polling has regular calibration events called "elections", and following those events biases and adjustment factors, most falling out of statistical analysis and correlations, rather than "gut feels", are updated.
The problem for the US seems to be that the biases are accelerating well in advance of those adjustments, as my own poll-vs-vote analysis shows:
Thread: <https://news.ycombinator.com/item?id=42060547>
Most robust analysis / results here: <https://news.ycombinator.com/item?id=42064420>
Polling companies are in the business of media deals and government contracts. They will develop methodology and reporting to that end and the money is in "a close and contested race", even if it won't be.
Of course, if we want to assess which methodology is more accurate, we'll need more than two cherry-picked examples.
Methodologically, prediction markets are nonrandom sampling from a pool of self-selected rich people (those with sufficient discretionary income to place bets), tempered by several influences, among which profit motive is only one. See the case of assassination markets for potential confounding factors:
<https://en.wikipedia.org/wiki/Assassination_market>
And from what I'm familiar with in other forms of gaming, gamblers often lose, and often bigly.
There is some research on the question:
David M. Rothschild, "Forecasting Elections: Comparing Prediction Markets, Polls, and Their Biases", January 2009, Public Opinion Quarterly 73(5). DOI:10.1093/poq/nfp082
<https://www.researchgate.net/publication/228306045_Forecasti...>
Joyce E. Berg, Forrest D. Nelson, Thomas A. Rietz, "Prediction market accuracy in the long run" International Journal of Forecasting, Volume 24, Issue 2, April–June 2008, Pages 285-300 <https://doi.org/10.1016/j.ijforecast.2008.03.007>
"A model incorporating markets that allow betting on elections suggests a role in prognostications" (31 October 2024) <https://anderson-review.ucla.edu/prediction-markets-polls-ec...>
Referencing Mikhail Chernov, Vadim Elenev, and Dongho Song, "The Comovement of Voter Preferences: Insights from U.S. Presidential Election Prediction Markets Beyond Polls" (28 October 2024) <https://anderson-review.ucla.edu/wp-content/uploads/2024/10/...> (PDF).
A pick of VP is not easy to model as some sort of stochastic variable from sample data.
Not the two betting examples.
Again: a systemic review is required, and I've linked some of the extant literature.
Also note that a "90% / 10% change to win" is not necessarily "wrong" if the 10% candidate wins. Anyone who has played an RPG will tell you that 90% chance to hit is far from certain. Maybe if there had been 100 elections, Clinton would have won 90 of them.
Messaging is build on focus groups, and tweaked to get the best results by both sides. That group is the same group that does polls.
Its a Goodhart's law in action: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
https://x.com/visegrad24/status/1853818243003125934
He put Trump's true probability of winning at 90% and a win of the popular vote at 75%.
Worth pointing out though that most pollsters _have_ been weighting by 2020 vote, so in general this isn't a fair critique of the entire polling industry. There are other fair critiques though, for example, that there are entire populations of people that are almost impossible to reach now (e.g. those who don't answer unknown numbers, young people, etc.).
[0] https://www.nytimes.com/2024/10/06/upshot/polling-methods-el...
Even if the true probability isn't 50% I doubt it's that far off.
Pollsters such as Nate Silver were giving gut-takes of Red over Blue, e.g.:
"Nate Silver: Here’s What My Gut Says About the Election, but Don’t Trust Anyone’s Gut, Even Mine" (Oct. 23, 2024)
<https://www.nytimes.com/2024/10/23/opinion/election-polls-re...>
I've done a somewhat half-assed take tonight of comparing actual returns to latest pre-election polling by state
Why that is, isn't clear. Political pollsters have been struggling for years with accuracy issues, particularly as landline usage falls (it's <20% in most states now), and unknown-caller blocking is more widely used (both on landlines and mobile devices).
Polling does have periodic calibration events (we call those "elections"), but whatever biases the polls seem to experience in the US, it's apparently systemically exceeding adjustment factors.
Polls / votes and deltas:
QC State EV BP RP BV RV Bd Rd
4: AL 9 36 64 32 65 -4 1
4: AK 3 45 55 0 0
4: AZ 11 49 51 49 50 0 -1
4: AR 6 36 64 34 64 -2 0
4: CA 54 63 37 60 37 -3 0
4: CO 10 56 44 55 43 -1 -1
4: CT 7 59 41 54 44 -5 3
4: DC 3 92 7 90 7 -2 0
4: DE 3 58 42 56 42 -2 0
4: FL 30 47 53 43 56 -4 3
4: GA 16 49 51 48 51 -1 0
4: HI 4 64 36 0 0
4: ID 4 33 67 33 64 0 -3
4: IL 19 57 43 52 47 -5 4
4: IN 11 41 57 39 59 -2 2
4: IA 6 46 54 42 56 -4 2
4: KS 6 42 51 41 57 -1 6
4: KY 8 36 64 34 64 -2 0
4: LA 8 40 60 38 60 -2 0
4: ME 2 54 46 0 0
4: ME-1 1 61 39 0 0
4: ME-2 1 47 53 0 0
4: MD 10 64 36 60 37 -4 1
4: MA 11 64 36 62 35 -2 -1
4: MI 15 50 49 0 0
4: MN 10 53 47 0 0
4: MS 6 40 60 37 62 -3 2
4: MO 10 43 57 42 56 -1 -1
4: MT 4 41 59 33 64 -8 5
4: NE 4 41 59 42 56 1 -3
4: NE-2 1 54 46 0 0
4: NM 5 54 46 51 47 -3 1
4: NV 6 50 50 0 0
4: NH 4 53 47 52 47 -1 0
4: NJ 14 57 43 51 46 -6 3
4: NY 28 59 41 55 44 -4 3
4: NC 16 49 51 48 51 -1 0
4: ND 3 33 67 31 67 -2 0
4: OH 17 46 54 44 55 -2 1
4: OK 7 33 67 32 66 -1 -1
4: OR 8 56 44 55 43 -1 -1
4: PA 19 50 50 0 0
4: RI 4 58 42 55 42 -3 0
4: SC 9 44 56 40 58 -4 2
4: SD 3 36 64 29 69 -7 5
4: TN 11 38 62 34 64 -4 2
4: TX 40 46 54 42 57 -4 3
4: UT 6 39 61 43 54 4 -7
4: VT 3 67 34 64 32 -3 -2
4: VA 13 53 47 51 47 -2 0
4: WA 12 59 41 58 39 -1 -2
4: WV 4 30 70 28 70 -2 0
4: WI 10 50 49 0 0
4: WY 3 73 27 70 28 -3 1
Blue votes: 43
Red votes: 43
Blue delta: -2.49
Red delta: 0.63
- QC: A parsing QC value (number of raw fields)
- State: 2-char state code, dash-number indicates individual EVs for NE and ME.
- EV: Electoral votes
- BP: Blue polling
- RP: Red polling
- BV: Blue vote return
- RV: Red vote return
- Bd: Blue delta (vote - poll)
- Rd: Red delta (vote - poll)
The last two results are the cumulative average deltas. Blue consistently performed ~2.5 points below polls, red performed ~0.6 points above polls.
Data are rounded to nearest whole percent (I'd like to re-enter data to 0.1% precision and re-run, though overall effect should be similar). Deltas are computed only where voting returns are >0.
Data are hand-entered from 538 and ABC returns pages.
Blue consistently polled slightly higher than performance. Polls don't seem to include third parties (mostly Green, some state returns include RFK or others).
There are all but certainly coding/data entry errors here, though for illustration the point should hold.
With updated (and 0.1% decimal precision) election returns, Harris's polling delta falls to -2.25% (Orange is unchanged). The overall advantage of her opponent over polling data is 2.89%. Which is a lot.
Still want to get more precise polling numbers in there, but again, it's not shifting a lot. Law of Large Numbers dictates that, as multiple rounded numbers tend to even out the precision distinction.
I've just re-run my analysis with higher precision on the deltas. Harris performed worse in every single race save DC than projected. Orange performed better in a majority of races, by as much as 5+ percent.
(I still need more accurate data for polling, I'll add a comment when I've updated that.)
State EV Poll (D/R) Vote (D/R) Delta (D/R) Win
AL 9 36.1 63.9 34.2 64.8 -1.9 0.9 R
AK 3 45.1 54.9 40.4 55.6 -4.7 0.7 R
AZ 11 49.0 51.0 47.2 51.9 -1.8 0.9 R
AR 6 35.7 64.3 33.6 64.2 -2.1 -0.1 R
CA 54 62.7 37.3 57.4 40.0 -5.3 2.7 D
CO 10 56.2 43.8 54.6 43.1 -1.6 -0.7 D
CT 7 58.7 41.3 54.5 43.8 -4.2 2.5 D
DC 3 92.4 7.6 92.4 6.7 0.0 -0.9 D
DE 3 58.1 41.9 56.5 42.0 -1.6 0.1 D
FL 30 47.0 53.1 43.0 56.1 -4.0 3.0 R
GA 16 49.4 50.6 48.5 50.8 -0.9 0.2 R
HI 4 63.7 36.3 62.2 36.1 -1.5 -0.2 D
ID 4 33.2 66.8 30.7 66.5 -2.5 -0.3 R
IL 19 57.4 42.6 53.3 45.3 -4.1 2.7 D
IN 11 41.4 58.6 39.2 59.1 -2.2 0.5 R
IA 6 46.4 53.7 42.3 56.3 -4.1 2.6 R
KS 6 41.9 58.1 40.8 57.4 -1.1 -0.7 R
KY 8 36.0 64.0 33.9 64.6 -2.1 0.6 R
LA 8 39.6 60.4 38.2 60.2 -1.4 -0.2 R
ME 2 54.3 45.7 53.1 44.3 -1.2 -1.4 D
ME-1 1 61.2 38.8 60.4 33.6 -0.8 -5.2 D
ME-2 1 46.9 53.1 45.0 52.9 -1.9 -0.2 R
MD 10 64.2 35.8 60.2 37.3 -4.0 1.5 D
MA 11 64.0 36.0 61.9 35.9 -2.1 -0.1 D
MI 15 50.6 49.4 48.2 49.8 -2.4 0.4 R
MN 10 52.9 47.1 51.1 46.8 -1.8 -0.3 D
MS 6 40.5 59.5 37.7 61.1 -2.8 1.6 R
MO 10 42.9 57.2 40.1 58.5 -2.8 1.3 R
MT 4 41.0 59.0 38.4 58.5 -2.6 -0.5 R
NE 4 41.3 58.7 38.5 60.2 -2.8 1.5 R
NE-1 1 41.6 58.4 42.4 56.3 0.8 -2.1 R
NE-2 1 53.5 46.5 51.2 47.5 -2.3 1.0 D
NE-3 1 22.6 77.4 22.5 76.3 -0.1 -1.1 R
NM 5 53.7 46.3 51.6 46.1 -2.1 -0.2 D
NV 6 50.0 50.0 46.8 51.5 -3.2 1.5 R
NH 4 53.0 47.0 51.0 48.0 -2.0 1.0 D
NJ 14 56.9 43.2 51.5 46.6 -5.4 3.4 D
NY 28 58.9 41.5 55.4 44.6 -3.5 3.1 D
NC 16 49.4 50.6 47.7 51.1 -1.7 0.5 R
ND 3 33.3 66.7 30.8 67.5 -2.5 0.8 R
OH 17 45.8 54.2 43.9 55.2 -1.9 1.0 R
OK 7 33.2 66.8 31.9 66.2 -1.3 -0.6 R
OR 8 56.5 43.6 54.9 42.5 -1.6 -1.1 D
PA 19 50.0 50.0 48.4 50.7 -1.6 0.7 R
RI 4 58.4 41.7 55.5 42.4 -2.9 0.7 D
SC 9 43.7 56.3 40.5 58.1 -3.2 1.8 R
SD 3 36.0 64.0 33.0 64.7 -3.0 0.7 R
TN 11 38.1 61.9 34.4 64.3 -3.7 2.4 R
TX 40 46.3 53.7 42.4 56.3 -3.9 2.6 R
UT 6 38.9 61.1 38.9 58.9 0.0 -2.2 R
VT 3 66.5 33.5 64.3 32.6 -2.2 -0.9 D
VA 13 53.4 46.6 51.8 46.6 -1.6 0.0 D
WA 12 58.9 41.1 58.6 39.1 -0.3 -2.0 D
WV 4 29.8 70.2 27.9 70.2 -1.9 0.0 R
WI 10 50.6 49.5 48.8 49.7 -1.8 0.2 R
WY 3 27.4 72.6 26.1 72.3 -1.3 -0.3 R
Blue votes: 56
Red votes: 56
Blue delta: -2.26
Red delta: 0.42
- Harris did more poorly than forecast in all but three races: DC, UT, and NE-1.
- Her opponent did better than forecast in 32 races.
- Many of Harris's bigger under-performances were in races she won, notably CA. FL and TX are losses with far worse-than-polled returns.
Net average polling bias is 2.68 points favouring the GOP across 56 contests.
Voting returns: <https://apnews.com/projects/election-results-2024/>
Polling data: <https://projects.fivethirtyeight.com/2024-election-forecast/>
So basically consistent with 2016 and 2020: Most polls have a 2-5 point bias in favor of Democrats. Maybe a bit improved from previous elections.
(and this was while Biden was still in the race)
People are looking at the popular vote and freaking out but lets not forget that there's still 7 million left to count in California and it's expected to net Harris almost 3 million votes
That pool was apparently more the former than the later.
Trump won many of those states by 2-3%.
Otherwise the real result would be distributed around the mean within the margin of error.
There is some bias and the polls did not correctly factor that into their statistical model.
This is a retweet of Nate Silver's analysis showing that the most probable outcome, with nearly a 25% chance, was Trump sweeping the swing states. Yes, added up with all the other possible outcomes, the race was very close, but this was still the most likely outcome, and by a large margin: https://x.com/LPDonovan/status/1854265746601488727
https://www.realclearpolling.com/polls/president/general/202...
It was no surprise he won, IMO.
Nate Silver nailed this in the 2016 election. He said Trump's victory there was consistent with historically normal polling errors.
What may have been less widely appreciated is these errors are not related to causes like limited sample size that are straightforwardly amenable to statistical analysis. They come from the deeper problems with polling and the way those problems shift under our feet a little bit with each election.