Since we launched the PunditTracker blog, our inbox has been inundated with the following request: “Track those NFL Draft pundits!” We understand and fully agree with this suggestion. Draft punditry is widely consumed but epitomizes the accountability gap in the prediction industry. Multiple years must pass before outcomes can be determined, and by that time, everyone is focused on the next draft. The old predictions are long forgotten.
We are working on a project to track the historical accuracy of pundit player rankings (a.k.a. Big Boards). It is a complex task, given that there is no standardized rating system for football players (e.g. how do we compare a defensive back to a running back?). We will have more to report on this project in the coming months and welcome any suggestions in the interim.
For now, let’s turn our attention to those ubiquitous mock drafts, whereby pundits predict which players will go to which teams. This is a more layered — and we would argue less interesting — prediction format, since pundits must devalue their own player rankings in favor of their views of each team’s needs and player evaluations.
We have dug up and graded the Round 1 mock drafts of Mel Kiper (ESPN), Todd McShay (ESPN), and Mike Mayock (NFL.com) from 2009 through 2011. We calculated a variability measure based on the difference between when each player was projected to be picked and when he was actually picked. A lower variability score is better.
The results tell a clear story: Todd McShay has issued the most accurate mock drafts for each of the past three years.
|McShay||8.7 (#1)||8.8 (#1)||6.8 (#1)||8.1 (#1)|
|Kiper||8.8 (#2)||10.4 (#3)||7.7 (#2)||9.0 (#2)|
|Mayock||9.6 (#3)||9.9 (#2)||9.3 (#3)||9.6 (#3)|
Here is a breakdown of the major hits and misses for the group:
Hits and Misses
We will revisit this post next month following the 2012 draft.
Update (4/22/13): We will be tracking these three pundits again for the 2013 NFL Draft. In the meantime, you have still have time to make some of your own Draft predictions on our website – click here to visit our Sports Page and use the “Vote Now” buttons.
- NFL Mock Draft Update: Todd McShay is #1 (Again) | PunditTracker
- McShay: Difference not big between RGIII and Tannehill | FTRsports.com | For The Record Sports