There has to be a piece to be written about Nate Silver and Ezra Klein--about how they both have performed Christensenian disruptions on pieces of their newspapers, election political coverage in the case of Nate Silver and domestic policy coverage in the case of Ezra Klein, about how both are the objects of whispering (and in Nate's case not-so-whispering) campaigns of scorn and fear, and about how neither of the newspapers they work for has done anything to try to generalize the experience…
Ezra Klein on Nate Silver, October 30, 2012:
The Nate Silver backlash: It seems to happen every few weeks. The race tightens or widens or simply continues on exactly as it’s been, and some pundit or reporter declares it a staggering humiliation for the burgeoning world of election quants.
Niall Ferguson took his turn at bat in early September. “The economy is in the doldrums,” he wrote. “Yet the incumbent is ahead in the polls. According to a huge body of research by political scientists, this is not supposed to happen.”
Actually, according to a huge body of research by political scientists, that was exactly what was supposed to happen.
After the first debate, David Frum stepped up to the plate. “Political science proclaims, ‘debates don’t matter,’ ” he wrote. “After this election, we may need to retire a lot of political science.”
Actually, “political science” never declared debates don’t matter. A political scientist — George Washington University’s John Sides — had reviewed the evidence and written that “when it comes to shifting enough votes to decide the outcome of the election, presidential debates have rarely, if ever, mattered.”
That was true then. It’s true now. It will be true if Mitt Romney wins the election (saying something has “rarely, if ever, happened” does not mean it cannot happen, as a look at the rather unusual weather outside your window will prove). And it will be true if President Obama, as continues to look slightly likelier than not, wins the election.
Which brings me to the backlash against Nate Silver.
Before we get too deep in the weeds here, it’s worth being clear about exactly what Silver’s model — and that’s all it is, a model — is showing. As of this writing, Silver thinks Obama has a 75 percent chance of winning the election. That might seem a bit high, but note that the BetFair markets give him a 67.8 percent chance, the InTrade markets give him a 61.7 percent chance and the Iowa Electronic Markets give him a 61.8 percent chance. And we know from past research that political betting markets are biased toward believing elections are more volatile in their final weeks than they actually are. So Silver’s estimate doesn’t sound so off.
Moreover, Silver’s model is currently estimating that Obama will win 295 electoral votes. That’s eight fewer than predicted by Sam Wang’s state polling meta-analysis and 37 fewer than Drew Linzer’s Votamatic.
So before we deal with anything Silver has specifically said, it’s worth taking in the surrounding landscape: Every major political betting market and every major forecasting tool is predicting an Obama victory right now, and for the same reason: Obama remains ahead in enough states that, unless the polls are systematically wrong, or they undergo a change unlike any we’ve yet seen in the race, Obama will win the election.
There’s no doubt about that. Real Clear Politics, which leans right, shows Romney up by 0.8 percent nationally, but shows Obama up in Ohio, New Hampshire, Iowa, Nevada, Wisconsin, Pennsylvania and Michigan. Romney is up in Florida and North Carolina, but note that his lead in Florida is smaller than Obama’s lead in Ohio. And RCP shows Colorado and Virginia tied. Pollster.com, meanwhile, shows Obama leading by a point in Colorado and Virginia and the race tied in Florida.
It’s important to be clear about this: If Silver’s model is hugely wrong — if all the models are hugely wrong, and the betting markets are hugely wrong — it’s because the polls are wrong. Silver’s model is, at this point, little more than a sophisticated form of poll aggregation.
But it’s just as important to be clear about this: If Mitt Romney wins on election day, it doesn’t mean Silver’s model was wrong. After all, the model has been fluctuating between giving Romney a 25 percent and 40 percent chance of winning the election. That’s a pretty good chance! If you told me I had a 35 percent chance of winning a million dollars tomorrow, I’d be excited. And if I won the money, I wouldn’t turn around and tell you your information was wrong. I’d still have no evidence I’d ever had anything more than a 35 percent chance.
There are good criticisms to make of Silver’s model, not the least of which is that, while Silver is almost tediously detailed about what’s going on in the model, he won’t give out the code, and without the code, we can’t say with certainty how the model works. But the model is, at this point, Silver’s livelihood, and so it’s somewhat absurd to assume he’d hand it out to anyone who asks. For better or worse, those aren’t rules we apply in other markets, or even in journalism, where off-the-record conversations inform much that journalists say.
Another criticism is that Silver’s model is so based on polls by the end that it barely actually counts as a model, and Silver is simply doing a savvy job repackaging and commercializing polling data that is out there already. This criticism, of course, says nothing about his accuracy.
Then there’s the argument that Silver adds unnecessary factors, though note that models without those factors, like Wang’s, are showing an even more lopsided win for the president.
But the good arguments over Silver’s model are being overwhelmed by very bad ones.
First, there are the conservatives who don’t like Silver’s model because, well, they don’t like it. Obama’s continued strong showing is prima facie evidence of bias. Or, to put it slightly differently, the model must be skewed.
The answer to this is simple enough: If Silver’s model is systematically biased, there’s a market opportunity for anyone who wants to build a better model. That person would stand to gain hugely if they outpredicted punditry’s reigning forecaster (not to mention all the betting markets and all the other forecasters). The math behind what Silver is doing isn’t that complicated and the polls are easily available. But so far, the most popular conservative take on the polls was UnskewedPolls.com, to which … LOL. If Silver’s model is so easy to best, then what’s the market failure keeping a less-biased source from besting it?
Then there’s the backlash from more traditional media figures. Some of the arguments here have been downright weird, as when Politico’s Josh Gerstein wrote, “Isn’t the basic problem with the Nate Silver prediction in question, and the critique, that it puts a percentage on a one-off event?” Or when Politico’s Jonathan Martin wrote, “Avert your gaze, liberals: Nate Silver admits he’s simply averaging public polls and there is no secret sauce.” Or when Politico’s Dylan Byers wrote, “So should Mitt Romney win on Nov. 6, it’s difficult to see how people can continue to put faith in the predictions of someone who has never given that candidate anything higher than a 41 percent chance of winning.”
Come to think of it, a lot of the odder critiques of Silver have been coming out of Politico. But that makes a kind of sense. Silver’s work poses a threat to more traditional — and, in particular, to more excitable — forms of political punditry and horse-race journalism.
If you had to distill the work of a political pundit down to a single question, you’d have to pick the perennial “who will win the election?” During election years, that’s the question at the base of most careers in punditry, almost all cable news appearances, and most A1 news articles. Traditionally, we’ve answered that question by drawing on some combination of experience, intuition, reporting and polls. Now Silver — and Silver’s imitators and political scientists — are taking that question away from us. It would be shocking if the profession didn’t try and defend itself.
More recently, we in the media — and particularly we in the media at Politico — have tried to grab an edge in the race for Web traffic by hyping our election stories far beyond their actual importance. The latest gaffe is always a possible turning point, the momentum is always swinging wildly, the race is endlessly up in the air. It thus presents a bit of a problem for us if our readers then turn to sites like Silver’s and find that none of this actually appears to be true and a clear-eyed look at the data shows a fairly stable race over long periods of time.
My guess is Silver and his successors will win this one, if only because, for all the very real shortcomings of models, election forecasters have better incentives than homepage editors. For instance, note that all these attacks on Silver take, as their starting point, Silver’s continuously updated prediction for the presidential election, which includes point estimates for the popular vote and electoral college, and his predictions for the Senate races. Those predictions let readers check Silver’s track record and they force Silver, if he wants to keep his readers’ trust, to make his model as accurate as he can. That’s a good incentive structure — certainly a better one than much of the rest of the media has — and my guess is his results, over time, will prove it.
