undefined | Better HN

0 pointsgh02t5d ago0 comments

The real caveat is that 538 was a Monte Carlo model, and is only as good as its inputs. "Here's what the current spread in polling numbers is *given our model and the current polling and their reported uncertainties.*" Polling uncertainties are themselves computed under certain models, and those models are subject to errors. I don't think 538 hid this, but it's a difficult caveat for people to reason about because the sorts of modeling errors that have the most influence usually represent "unknown unknowns".

0 comments

this_user5d ago

Building a model for predicting the ultimate winner of a US presidential election is particularly difficult, because you are dealing with noisy input data and nonlinear effects, i.e. just a few thousand votes in a few key states can completely flip the outcome. If you then have poorly calibrated polls with a large margin of error, there is really nothing much you can do.

On the other hand, it does raise the question how valuable the 538 models for something like this really are if the outcome is a coin flip anyway.

ngriffiths5d ago

Exactly, and correlated errors, where a polling error in one state predicts similar errors across the board.

I disagree that it's all pointless though. Most basically it's smart for campaigns to have a good model and let that inform strategy where appropriate. Since the president is a big deal other people's decisions are also impacted, and in the long run it pays to have good predictions of those chances. Also, the outcome sometimes is fairly certain and that isn't always easy to see.

gh02tOP5d ago

I agree, it's far from pointless. The 538 model is arguably close to the best you can do considering how difficult the task is, but it's important to understand it as purely a reflection of the polling data (and 538's reliability scores for polls), and that polling data is inherently flawed. After all, there are only 2 ways to perform a perfectly accurate poll: either know the outcome a priori, or run the election. We shouldn't be too surprised when models like 538 fail to correctly predict the outcome, because that's not what they represent. It's an analytical tool for understanding the current state of polling.

bryanlarsen5d ago

Regularly referring to that ~30% spread as "one polling error" made this a lot more understandable than most statistics for many people.

j / k navigate · click thread line to collapse