And the Oscar for Best Pundit (2012) goes to. . . .

February 27, 2012  |  Entertainment

Following the conclusion of the 84th Annual Academy Awards last night, it seemed only natural to turn our attention to the pundits who make their living predicting which Hollywood stars will come home with the world’s most infamous little gold statuettes. We hope the data below will prove informative, especially for those who spend countless hours trying to win their Oscar Pool with all kinds of techniques that range from picking the winners of the other awards shows to avoiding nominees that have already won. We would submit that the best technique may simply be to follow the pundit who has demonstrated time and again the ability to pick the most winners.

2012 is our first step in cataloguing those records. We chose 21 well-known film pundits, each of whom made predictions for all the Oscar categories.

Here are the rankings by hit rate (# of correct calls divided by # of total calls):

Hit Rate

Clayton Davis, The Awards Circuit         83.3%
Melena Ryzik, New York Times         83.3%
Kris Tapley, HitFix         79.2%
Peter Knegt, Indiewire         79.2%
Glenn Whipp, The Envelope         79.2%
Vegas favorites         77.1%
Pete Hammond,         75.0%
Steve Pond, TheWrap         75.0%
Sasha Stone, Awards Daily         75.0%
Dave Karger, Entertainment Weekly         75.0%
Anne Thompson, Indiewire         70.8%
Guy Lodge, HitFix         66.7%
Kyle Buchanan, Vulture         66.7%
Scott Feinberg, Hollywood Reporter         66.7%
Kevin Polowy, NextMovie         66.7%
Lou Lumenick, New York Post         65.2%
Jim Slotek, Toronto Sun         58.3%
Rotten Tomatoes         58.3%
Mark Harris, Grantland         58.3%
Richard Roeper, Chicago Sun-Times         58.3%
Erik Childress,         54.2%


These results are decidedly mediocre. Simply picking the favorites as deemed by Las Vegas oddsmakers (a proxy for crowd consensus) would have yielded a higher hit rate than 16 of our 21 pundits.

As we have discussed in previous posts, we believe the hit rate approach is flawed. All calls are not equal and should not be scored as such. Instead, pundits should receive more credit for predictions that are out-of-consensus. For instance, Christopher Plummer was a virtual lock for Best Supporting Actor, and so that pick should result in less credit than, say, choosing “Hugo” for Cinematography (over the favored “The Tree of Life”). Our “$1 Bet Yield” metric calibrates predictions for boldness by measuring the average payout (using Vegas odds) had you placed $1 bets on each of the pundit’s selections. Here is how the Oscar pundits stacked up using this approach:

$1 Bet Yield

Kris Tapley, HitFix         $1.39
Peter Knegt, Indiewire         $1.37
Clayton Davis, The Awards Circuit         $1.35
Melena Ryzik, New York Times         $1.35
Pete Hammond,         $1.22
Glenn Whipp, The Envelope         $1.16
Steve Pond, TheWrap         $1.14
Sasha Stone, Awards Daily         $1.11
Vegas favorites         $1.10
Dave Karger, Entertainment Weekly         $1.09
Anne Thompson, Indiewire         $1.07
Guy Lodge, HitFix         $1.01
Kyle Buchanan, Vulture         $1.01
Lou Lumenick, New York Post         $0.98
Jim Slotek, Toronto Sun         $0.98
Scott Feinberg, Hollywood Reporter         $0.96
Kevin Polowy, NextMovie         $0.95
Rotten Tomatoes         $0.83
Mark Harris, Grantland         $0.76
Richard Roeper, Chicago Sun-Times         $0.73
Erik Childress,         $0.63
S.T. VainAirsdale, MovieLine         $0.59


Here, the pundits fare a bit better, thanks largely to underdog picks such as “Undefeated” in Documentary and “Hugo” in both Cinematography and Visual Effects. Still, the performance is uninspiring, with less than 40% of the group outperforming Vegas. 

Next year, we plan to track the awards predictions earlier in the process. The other award shows (Screen Actors Guild, Golden Globes, etc) tend to make the market more efficient, which in turn makes it more difficult to generate high Bet Yields come Oscar night, particularly in the major categories.


Update (2/27/12): Added one more pundit, Melena Ryzik of the Carpetbagger blog (New York Times).




Leave a Reply

  1. Have a pundit you would like us to track? Or some general feedback on the site? Let us know!