We consider the bible of pundit prediction analysis to be Philip Tetlock’s Expert Political Judgment. The book details an epic study conducted by Tetlock, in which he tracked more than 80,000 predictions from nearly 300 political pundits over the course of 20 years.
There are plenty of fascinating insights in the book, but we particularly enjoy the roster of excuses that pundits trot out when confronted with the fact that their prediction was dead wrong (Tetlock refers to these as “belief system defenses”). Here are some of our favorites:
Pundit defenses (a.k.a. lessons in how to not admit that you were wrong)
Too early: “I was simply too early; just wait and see, that stock market crash is still coming.” (see: Broken Clock Pundits)
Black swan: “Sure, our credit rating models failed, but who could have predicted that housing prices would fall across the country at the same time?”
Close enough: “Hey, I said the stock market would go up more than 10% and it went up 8%. I was basically right.”
Self-negated: “It was our own beliefs and actions that spared the world from catastrophe.” (see When Prophecy Fails)
Hedged: “I only said that it could happen.” (See: The 40% Rule) — note: when pundits are correct, they strangely fail to mention the hedge.
What other excuses have you heard? And can you think of any specific examples of the dodges above?
Friendly reminder: Only a few days left to make your predictions on the 2013 calls from Byron Wien, Karl Rove, Doug Kass, and others.