Hacker News Reader

Comment by joshdavham | Hacker News Reader

I always get annoyed when people look at election outcomes and say “the polls were wrong” when the most likely outcome didn’t happen. It’d be like saying there’s a 5/6 chance that a 6-sided dice will not roll a 4, rolling a 4, and then concluding that the initial proposed probability was wrong.

View on HN

That's not how polls work though. A poll shows how a sample of people say they'll vote. An ideal poll would have the same proportions as the final popular vote count.

Thank you! Lordy it's crazy that this thread isn't distinguishing polls which are a measurement and the resulting prediction based on those polls.

If you polled 60%/40% yea/nay you wouldn't give the nays a 40% chance of winning. You need P(yeas > 50%) based on your observed measurement.

I came away from 2016 wondering if any of the electoral outcome stuff is worth doing given how much it gets misunderstood.

“They said Hillary had a 90% but Trump won! They’re idiots!”

No! They said Trump had a 10% chance and he won. It was a predicted outcome!

The problem is using statistical models at all for predicting the probability of one time events. (elections are every 4 years, but those future models will be different).

You can't validate such a model, and it's not useful since any upset victory was "accounted for" simply because its possibility was named.

538 had Trump at 30.7% the night before, FWIW: https://web.archive.org/web/20161107224336/https://projects....

But is your 10% chance of Trump winning model a better model than one that gave Trump a 40%, 50%, 60%, 70%, 80%, or 90% chance of winning?

If you can't show that why should anyone listen to you?

It is a rational behavior to question a prediction if an event that was deemed to have low probability actually happened, I don’t like the implication that it is stupid to do so.

In fact, after thorough investigation, most polling agencies did identify significant factors that let to bias, and almost all major pollsters have adapted their methods in later elections as a result: education weighting, compensating for nonresponse bias, diversified polling channels beyond phone calls (online surveys, text messages, in-app polling), likely-voter modeling…

I don’t think any professional pollster will tell you that the prediction you mentioned was reasonably accurate in retrospect, regardless of affiliation.

I don’t attribute malice or incompetence, it’s clear that the game changed suddenly and new confounding factors became significant. People that would previously vote together diverged significantly, lots of politically inactive people became involved, people were afraid to be honest, and people that were harder to reach were voting much differently to those that were more accessible to pollsters. That will really mess with any sampling strategy that proved reliable for many years before.

If someone says there is an x% chance of an event happening, is there a value of x other than 0 & 100 where they can be wrong?

What is the value that someone can derive out of such predictions ?

If someone predicts something will happen with a 90% probability, then they should be wrong roughly 10% of the time. We can't determine that from a single event, but we could look at 10 events predicted by the same model. In the case of election forecasts, we would expect 1/10 elections predicted at 90% probability to have the wrong estimate. So we would expect the average person to see it at least once, and probably twice, in their lifetime.

Reply on news.ycombinator.com