> What are you even classifying as accurate or correct?
When somebody gives a prediction of the outcome of an election? I classify it as correct if they predicted the candidate who wins.
> Do you take every 51% prediction from FiveThirtyEight and if the result is a win you consider that forecast accurate? And every 49% prediction must result in a loss? This just not how statistical forecasts work.
No, but it is the way to map statistical forecasts to reality. He was quite explicitly predicting the outcome of the actual election. That prediction was incorrect.
The whole rating of the accuracy of these models is really snakeoil dressed up as science. There is a lot less rigorous science and a lot more feelings and adjusting numbers and twiddling formulas retrospectively than you were probably led to believe.
Would a 99-1 for Trump model have been worse or less accurate than a 51-49 for Clinton model? Despite predicting the correct outcome whereas the Clinton model predicted the incorrect outcome?
> I don't know why you're couching that in a hypothetical, FiveThirtyEight has repeatedly done that exercise.
Not really with much rigor. Where are their reproducible published papers and data sets? They made their name with a bit of luck on a fairly predictable election, but were unable to show a significant advantage in their methods across a number of elections.
> It is core to the concept of a probability that the result is going to go the opposite way from the prediction sometimes! It's meaningless to call it "wrong".
No no, that's not true. There are two different things here. Firstly, if you had a model and method of predicting elections that you applied to a sample of elections and showed that it had a good ability to correctly predict, then you can say your model is a good prediction across typical elections. The model getting one wrong does not make it a bad model over a set of elections. It absolutely is wrong for that particular election though. And secondly if you use a model to make a prediction about a particular election, when your prediction turns out to be wrong, it was not retroactively correct because it just followed the model and you claim the model is good. That's just not how statistics or predictions work.
