Monday, November 12, 2012

Nate Silver may be overrated, but poll aggregation isn't

tl;dr version: It's not exactly clear whether or not Nate Silver is the wunderkind he is made out to be (his true gift may be more in the explaining than in the predicting), but poll aggregation is an idea whose time has come -- and pundits and pols ignore it at their peril.

Let's get one thing out of the way up front: I really like Nate Silver. His writing style is pitch perfect, and people like me really get off on his blend of scientific humility and technical swagger. Independent of anything else, his mini-celebrity status is well-deserved, in my opinion.

Still, there are many who have questioned whether his "special sauce" is really contributing anything of value. There are even people whom I really respect that are quite down on Silver. I think this all raises some fair questions. Despite Silver's legendary performance in 2008 and 2010, and even despite the fact that he continued the streak by making (by some measures) the most accurate predictions in 2012, there is still room to argue that some of this is just statistical flukiness.

To be sure, while Silver was arguably the most accurate quant on this most recent election, his advantage over other poll aggregators was razor-thin. I consider it an open question whether the "special sauce" really does yield a consistently more accurate answer than other simpler approaches, or if Silver has just gotten a little lucky. But the thing is, it doesn't matter all that much.

Whether Silver is really doing something special, or if one can make do with a more streamlined approach that simply seeks to aggregate polling data in a sensible way, without all of the meticulous adjustments performed by the 538 model, one thing is perfectly clear: In the last several election cycles, poll aggregators have yielded consistently accurate results, even while blowhard pundits have floundered.

I think Silver's predictive success (as opposed to his professional success, which as I have mentioned I think is largely due to his excellent writing style and explanatory abilities) is in large part an inevitable result of the vast increase in the amount of independent state-level polling being done these days. While Darren Sherkat is correct when he says, "on average, if you heap together shit, it doesn’t equal filet mignon. The Central Limit Theorem is not something to be fucked with...", it now appears (if you'll allow me to extend the metaphor) that there is enough meat in modern state-level polling to at least make a decent hamburger, if not filet mignon.

There's a bit of an epistemological Catch-22 here, in that the low sample size will always give the doubters enough wiggle room to argue that it's all a fluke. But it really seems to me at this point that the most parsimonious explanation for Silver's continued streak, as well as the excellent performance of his quant counterparts, is that there is a core of legitimacy to poll aggregation, and that models like the one employed by Silver (if not his specific model) should now be recognized as far and away the most reliable method for predicting the results of an election.

There's still plenty of room to doubt some of Silver's more exotic "voodoo", but it is increasingly hard to deny that the basic idea is sound: Take all of the data that you have available, calibrate it to take into account each polling outfit's historical performance, and average it all together. Apply error bars of appropriate size (e.g. Silver should not get credit for calling 50 out of 50, since his 50.3% "prediction" on Florida hardly counts as "calling it"), and you now have a stable and reliable and sufficiently hedged prediction. How many more elections will it take before we can agree on this?

3 comments:

  1. Want to deal with the price manipulation issues for Binance? What are you looking for? Just dial Binance support number from your phone and get in touch with the professionals immediately. The experts will carefully examine your issue so that they can finalize the result-driven solutions immediately to the user so that issue get Binance Support Number resolved instantly. Lose the burden of queries by take time accessible solutions. They are approachable every time for the assistance so you can try their services as per your desire

    ReplyDelete
  2. Are you facing the issue while registration of Blockchain issue? Verification process is important as you get the access of various facilities. To get your verification done in one go, you can dial Blockchain support number and get in touch with the experts immediately. The experts immediately frame solutions and Blockchain Support NUmber methods on fingertips. Sometimes they also offer assistance to the users in stepwise manner for better understanding. So, contact them for eradicating the errors completely.

    ReplyDelete
  3. Are you facing the issue while registration of Gemini issue? Verification process is important as you get the access of various facilities. To get your verification done in one go, you can dial Gemini support number and get in touch with the experts immediately. The Gemini Support Number experts immediately frame solutions and methods on fingertips. Sometimes they also offer assistance to the users in stepwise manner for better understanding. So, contact them for eradicating the errors completely.

    ReplyDelete