Deadline Oscars

UPDATED WITH MORE DETAILS: The Academy Awards are important in and around Hollywood for primarily three reasons: the nominations which bring audiences into theaters as a movie marketing tool, the lucrative ‘For Your Consideration’ ads they generate, and the global telecast announcing winners so everybody can bask in their reflected glory. Now ABC is trying to corner the market on all with one move. Not only does the network broadcast the Oscars but its news division is guaranteeing data guru Nate Silver a role. How much of a Hollywood game-changer will this become? Not much of one judging from how little attention his movie awards prognostication has garnered in the past. Twice before, in 2009 and 2011, he sought to predict the Academy Award winners in 6 major categories based on a “mix of statistical factors”. His track record was 9 correct picks in 12 tries, for a 75% success rate. “Not bad, but also not good enough to suggest that there is any magic formula for this,” he wrote. For the 6 marquee categories he hadicapped in 2013, he was correct only for sure-things and missed the 2 that were more complex to predict. Meh.

I’ve been pondering this news scooped by Politico’s Mike Allen about all the  inducements ESPN/ABC News gave the 35-year-old to leave The New York Times, including extensive air time, a digital empire, and inclusion in the Oscars. A lot of showbiz websites and blogs large and small, smart and smarmy, clued-in and clueless, depend on their Oscar prognostication to drive traffic and foot bills. But unless Silver allows for the myriad variables that go into Academy Award noms and wins – insider stuff that Deadline knows from covering movie awards season in-depth – he won’t become more accurate.

For instance: Who’s popular, deserving, and appropriately humble enough to get nominated? Which film’s director is considered a douchebag whom nobody wants to win? What studio did a lousy job campaigning for the Academy Awards? How badly is Harvey Weinstein badmouthing the competition? I’ve always said that most Oscar voters are not just geriatric and cranky but also jealous and vengeful. Whether Silver’s statistical model can take into account those indiosyncracies and also cover more Oscar categories than just 6 is yet to be seen. But I’ll bet on Deadline’s own awards columnist Pete Hammond to beat Silver’s prognostications in 2014.

Obviously, the annual Academy Awards process isn’t as big a deal as U.S. national election campaigns. But interesting to note that Silver’s FiveThirtyEight blog was driving 20% of all traffic to the NYT as the last election electrified. That’s because in 2012 he correctly predicted the winners of all 50 states, in 2008 the winners of 49 out of 50 states, and the winners of all 35 U.S. Senate races that year as well. What ESPN/ABC offered was to return Silver to his flagship FiveThirtyEight.com and put him on air at ESPN and ABC, and develop verticals on a variety of new topics. And now he’ll work for the TV home of the Oscars. Odds are certain that Silver’s blog now will become one of the go-to places for Oscar dollars. But not for accuracy.

Can Silver truly become a trusted player in this showbiz space? Maybe. But he’ll have to do a lot better. Of course, if he’s wrong his first time out after being hyped way more than in the past, he’ll be laughed out of the biz. First, he has to stop relying on all the other film awards each year. They simply don’t matter. It might help if the Academy Of Motion Picture Arts & Sciences hands Silver its list of voters. Considering that AMPAS and ABC are joined at the hip because their broadcast pact goes at least through 2020, that’s doable. Whether or not the membership will resent having their privacy violated or participating in any polling is an impending challenge. Certainly the Academy over the years has discouraged voters from cooperating with any prediction schemes.

So what methodology will Silver use? As best as I can understand it (and, please remember that I’m mathematically challenged), it’s a so-called ’educated and calculated estimation’ stemming from his reliance on statistics and study of performance, economics, and metrics. This guy first developed the Elo rating for Major League baseball, a system that calculates the relative skill levels of players. He then developed his PECOTA system for projecting performance and careers and sold it. His FiveThirtyEight is a self-created political polling aggregation website (which took its name from the number of electors in the U.S. Electoral College) using a calculated model. He needs to better adapt that to the Oscars instead of just relying on other awards shows.

Here’s what Silver wrote about his Oscar predictions in 2013:

This year, I have sought to simplify the method, making the link to the FiveThirtyEight election forecasts more explicit. This approach won’t be foolproof either, but it should make the philosophy behind the method more apparent. The Oscars, in which the voting franchise is limited to the 6,000 members of the Academy of Motion Picture Arts and Sciences, are not exactly a democratic process. But they provide for plenty of parallels to political campaigns.

In each case, there are different constituencies, like the 15 branches of the Academy (like actors, producers and directors) that vote for the awards. There is plenty of lobbying from the studios, which invest millions in the hopes that an Oscar win will extend the life of their films at the box office. And there are precursors for how the elections will turn out: polls in the case of presidential races, and for the Oscars, the litany of other film awards that precede them.

So our method will now look solely at the other awards that were given out in the run-up to the Oscars: the closest equivalent to pre-election polls. These have always been the best predictors of Oscar success. In fact, I have grown wary that methods that seek to account for a more complex array of factors are picking up on a lot of spurious correlations and identifying more noise than signal. If a film is the cinematic equivalent of Tim Pawlenty — something that looks like a contender in the abstract, but which isn’t picking up much support from actual voters — we should be skeptical that it would suddenly turn things around.

Just as our election forecasts assign more weight to certain polls, we do not treat all awards equally.

Read More »