After announcing the idea of PunditTracker early in 2012, we solicited suggestions from our readers on which pundits to track. The most requested financial pundit, by far, was Jim Cramer. In fact, during our interview with NPR last March, host Bob Garfield ended the discussion with the not-so-subtle request: “Two words: Jim Cramer”.
We are making Cramer the first in our PT Focus series, in which we take an in-depth look at a pundit’s track record. For those who are unaware, Cramer is the host of the popular show Mad Money on CNBC, providing general financial advice as well as specific stock recommendations.
Cramer currently has an F grade on PunditTracker.com. Let’s walk through how we arrive at that score.
First, here are the parameters by which we evaluate Cramer’s stock recommendations:
- We started tracking his picks on January 1, 2011.
- We score only his “Featured” picks on Mad Money and not those made in other segments such as the “Lightning Round.” Our reasoning is that the Featured picks are unsolicited recommendations for which Cramer has presumably done more research than for picks given as live responses to viewer questions.
- We measure the performance of his picks relative to that of the S&P 500 index over the corresponding period.
- We assume a three-month holding period, unless Cramer reverses his stance on a given name (e.g. says Buy XYZ and then says Sell XYZ within the three months), in which case we “close out” the original recommendation. This holding period is based on the idea that Cramer tends to revisit his picks each quarter. (Note: We also have calculated performance using a six-month holding period; we are happy to provide the data if there is interest).
- The baseline stock price is the opening price two days after the recommendation is made, in order to account for any day-one “Cramer bump” effect.
- The hit rate is the percentage of Cramer’s picks that outperform the index. We equate sell ratings to short recommendations (i.e. they are scored as correct if the stock underperforms the S&P).
With the parameters out of the way, let’s now delve into the details of Cramer’s performance.
Given our assumed three-month holding period, we have now graded two years worth of Cramer’s picks: those made from January 2011 through December 2012. That amounts to 552 calls overall, of which 254 outperformed the index (46% hit rate).
On average, Cramer’s picks returned -0.08% versus the 1.35% S&P 500 return over the corresponding period. That amounts to 142 basis points of quarterly underperformance, or 568 basis points on an annualized basis, which amounts to an F grade in our grading system. (We award an A for 500+ basis points of annual equity outperformance and an F for 500+ basis points of underperformance).
Let’s now break down his results by year and by quarter:
|Cramer's Picks (three-month holding period)||-1.6%||1.1%||-0.1%|
|Relative performance (basis points)||-207||-89||-142|
|Hit Rate (% that outperformed)||48%||44%||46%|
Here were Cramer’s ten best and worst picks during the two years:
10 Best Picks
10 Worst Picks
Cramer underperformed the index on both his Buy and Sell recommendations, although his sells did even worse, with the stocks rising 4.9% on average versus 3.2% for the S&P.
Buy Sell (Short)
# of Picks 494 122
Cramer 1.1% -4.9%
S&P 2.4% -3.2%
Relative -135 -173
We will provide quarterly updates on Cramer’s performance going forward.
Can you pick stocks better than Jim Cramer? For those new to PunditTracker, we allow users to compete with pundits. Click here to visit our Finance page and start making predictions on various financial events, including Apple’s quarterly performance tomorrow (using the “Vote Now” buttons).
When we started tracking James Altucher in 2011, it didn’t take long to recognize that Altucher swings for the fences with his predictions. In fact, all seven of his predictions that we have cataloged have had Medium-High or High boldness ratings. Altucher’s approach is likely to result in a low Hit Rate, but his Yield will tell the story over the long run.
Three of his predictions have come due so far in 2013:
Dow Jones Index will hit 20,000 within one year (made on 2/27/12): Predictions don’t get much bolder than this — consider that the Dow was at 13K at the time — and this one proved to be wildly off the mark, with the Dow finishing February at just above 14,000.
Apple will be $1 trillion dollar company in one year (made on 2/6/12): Altucher was calling for more than a double in Apple’s valuation, and when the stock soared to $700 per share last fall, the prospect suddenly seemed plausible. However, the stock has since given back its gains and closed Feburary 6 at about $457 share, slightly down from when Altucher made his prediction.
S&P 500 to hit 1500 by early 2013 (made on 9/23/11): The S&P was at 1130 when Altucher made this prediction for a 30%+ return over the next sixteen months. This bold call was proven correct when the S&P eclipsed 1500 in late January.
Overall, Altucher has been 1-for-6 with the predictions we have tracked, with a $0.92 yield. Click here to see the details for all his calls, including yet another super-bold one: that Facebook will have a $300 billion market capitalization by 2017.
Brill’s Content, an old media-watchdog magazine, used to track the predictions of political pundits such as George Will and Eleanor Clift. The magazine incorporated an interesting twist: it compared the accuracy of these professional pundits with that of Chippy, a chimpanzee who chose random ‘yes’ or ‘no’ answers. Chippy did quite well, hitting on more than half of his “predictions” and beating out several of the experts.
Inspired by Chippy, today we are introducing three new pundits on PunditTracker: Groundhog Gary, Coin-Flip Chloe, and RNG Roger. These fictitious pundits will make one random prediction each week: Gary in Finance, Chloe in Politics, and Roger in Sports. Each Monday, we will take one popular topic in each category and then literally flip a coin to pick which side the pundit takes.
Not only will this feature allow us to compare the ‘experts’ with a random number generator, but it will also supply a steady stream of predictions for you to vote on.
Gary, Chloe, and Roger have alluredy made their predictions this week – click on their profile pages to make yours:
We consider the bible of pundit prediction analysis to be Philip Tetlock’s Expert Political Judgment. The book details an epic study conducted by Tetlock, in which he tracked more than 80,000 predictions from nearly 300 political pundits over the course of 20 years.
There are plenty of fascinating insights in the book, but we particularly enjoy the roster of excuses that pundits trot out when confronted with the fact that their prediction was dead wrong (Tetlock refers to these as “belief system defenses”). Here are some of our favorites:
Pundit defenses (a.k.a. lessons in how to not admit that you were wrong)
Too early: “I was simply too early; just wait and see, that stock market crash is still coming.” (see: Broken Clock Pundits)
Black swan: “Sure, our credit rating models failed, but who could have predicted that housing prices would fall across the country at the same time?”
Close enough: “Hey, I said the stock market would go up more than 10% and it went up 8%. I was basically right.”
Self-negated: “It was our own beliefs and actions that spared the world from catastrophe.” (see When Prophecy Fails)
Hedged: “I only said that it could happen.” (See: The 40% Rule) — note: when pundits are correct, they strangely fail to mention the hedge.
What other excuses have you heard? And can you think of any specific examples of the dodges above?
Friendly reminder: Only a few days left to make your predictions on the 2013 calls from Byron Wien, Karl Rove, Doug Kass, and others.
We are excited to announce that you can now easily share your predictions on social media sites such as Facebook. Next to each prediction on your My Profile page, you will find a Share link that provides a snapshot of your prediction — which we have framed in a competitive user-versus-pundit manner — with an embedded link for Facebook sharing. Comments on your prediction are also enabled.
Let the sharing / bragging / trash talking begin!
With Apple’s quarterly report coming out on Wednesday afternoon, today is the last day to cast your votes on Gene Munster’s predictions (revenue, earnings, iPhone shipments, Mac shipments, and iPod shipments).
Update (1/21/13): Post has been updated for Full Year 2012 Returns.
As we have discussed before, following the investment picks of the annual Barron’s Roundtable has been a lucrative approach over the years. From 2002-2011, the average Roundtable return was 11.5% versus -0.2% for S&P 500, with all but one of the members outperforming the index. From 2005-2011, those figures were 8.6% and 0.1%, respectively.
Let’s look at the individual Roundtable participants.
|Abby Joseph Cohen||2.2%||17.6%||8.4%||-18.9%||13.3%||15.1%||-2.7%||4.3%|
* returns only represent price appreciation, assumes an equal allocation across the picks; does not include recommendations from the mid-year roundtable; fixed-income securities are not evaluated
For a full breakdown of returns since 2002, click here.
2011 was the first year of underperformance for the group (-4.9% vs -1.1%). Was this the start of a trend, or would the group bounce back in 2012? Here is how the pundits fared in 2012
|Return||Best Pick||Worst Pick|
|Mario Gabelli||24.2%||Cincinnati Bell (+73%)||Navistar (-44%)|
|Oscar Schafer||19.9%||United Rentals (+54%)||Verint (+7%)|
|Marc Faber||19.6%||Wing Tai Holdings (+89%)||Short: Salesforce.com (-66%)|
|Brian Rogers||14.0%||Thermo Fisher Scientific (+34%)||Microsoft (-5%)|
|Abby Joseph Cohen||13.5%||JPMorgan Chase (+24%)||ExxonMobil (+2%)|
|Meryl Witmer||10.8%||Packaging Corp of America (+49%)||Tronox (-27%)|
|Scott Black||6.4%||CBS (+37%)||Apache (-19%)|
|Felix Zulauf||-5.0%||Buy: Gold (+4%)||Short: MSCI Emerging Market Index (-16%)|
|Fred Hickey||-8.9%||BMC Software (+24%)||Marvell Technology (-54%)|
For a full breakdown of the 2012 picks, click here.
The pundits collectively slightly underperformed the S&P again in 2012 (10.5% vs 11.6%). Mario Gabelli was the top performer, thanks to several 65%+ gainers: Fortune Brands Home & Security, Cincinnati Bell, and Gaylord Entertainment. Oscar Schafer’s recommendation of United Rentals propelled him to second place, while Marc Faber continued his spectacular performance, with his Singapore and Hong Kong bets paying off in spades. Meanwhile, Abby Joseph Cohen and first-time Roundtable member Brian Rogers both turned in ~14% gains. On the negative side, Felix Zulauf — who had been the best of the group through 2011 — has had his first sub-par year, as his bearish macro take has failed to play out yet in the markets. Finally, Fred Hickey’s recommendation of Marvell Technology has sunk him to the bottom of this year’s rankings.
Here are the updated compounded annual return figures for the periods 2002-12 and 2005-12 (Hickey joined in 2005), with Faber now edging out Zulauf over both time periods:
Updated CAGRs (incorporating 2012 figures)
|Abby Joseph Cohen||2.9%||5.4%|
To see all the individual picks of the Roundtable members as well as their letter grades, visit the PunditTracker Finance section. We will of course update all these figures once the final numbers are in.
We are now tracking more than 120 pundits on PunditTracker.com. With the 2013 Predictions rolling in, check out all the latest predictions and enter the PT Challenge to compete with the “experts” and win prizes. If you have suggestions for new financial pundits for us to track, please let us know.
Byron Wien recently released his predictions for 2013. We’ve always admired the creativity and breadth of his calls, which may be one explanation for a rather disappointing track record since we’ve been following him since 2010. With only 6 of 24 calls correct over the past two years and a yield of just 25%, here’s to Byron having a very happy and hit filled 2013.
You can vote on Byron’s active calls here:
After a flurry of activity surrounding our site launch and the election, we thought this would be an appropriate time to take a step back and revisit what we are trying to accomplish with PunditTracker.
The way we see it, we have two overarching goals: deflating inflated pundit reputations and surfacing the good pundits.
(1) Deflating inflated pundit reputations
Few would dispute the notion that there is an accountability gap in the world of punditry. Simply put, the incentives in the industry are all wrong, which results in hits (correct predictions) being reported far more frequently than misses (incorrect predictions).
Let’s assume the role of each industry participant to illustrate why this is the case:
- Pundit: When wrong, keep quiet. When right, self-promote: write a book, go on the speaking circuit, etc. Easy, no?
- Media: Pundit comes on your program and makes a bold prediction that turns out laughably wrong. What should you do? If you point out that fact, you indirectly discredit yourself. Instead, introduce the pundit again as an “expert”, with no reference to the wrong call. Moreover, consider how news is generated. Some event happens, which triggers you to sift through history to see who got it right. The flip side does not hold: there is no corresponding consideration for things that didn’t happen and a search for those that got it wrong saying it would.
- Us: Not only are we force-fed a skewed sample of prediction outcomes, but we are also psychologically wired to remember hits more than misses. As we discussed in a previous post, unusual information has an outsized grip on our memory. Bold calls are typically incorrect, so we quickly forget those. Meanwhile, bold calls that turn out right are unusual and therefore stick in our mind. And because we tend to confuse ease of recall with frequency, we develop a warped sense of the pundit’s batting average.
All these factors have given rise to what we call the pundit playbook. Pundits are entirely incentivized to churn out a bunch of bold predictions, knowing that there is plenty of upside if they get one right and no downside if they get them all wrong. Ever wonder how One-Hit Wonders and Broken Clocks are able to sell books about their new predictions despite pathetic track records?
PunditTracker aims to fix this moral hazard by playing the role of public scorekeeper. There are two notable twists in how we score predictions. Our first twist is to incorporate boldness in addition to accuracy. The typical “hit rate” or “batting average” approach (# of correct calls divided by # of total calls) assumes all predictions are equal, which is decidedly not the case. The daily prediction “the sun will rise tomorrow” would (hopefully) yield a perfect hit rate, after all. We instead use a “$1 Yield” metric, which measures the average payout had you bet $1 on each of the pundit’s predictions, based on consensus odds at the time (odds are driven by user votes). A yield of exactly $1.00, for instance, means the pundit’s predictions were no better or worse than the consensus view at the time.
(2) Surfacing the good pundits (assuming they exist)
A fundamental problem PunditTracker faces is that the pundits we are currently tracking are likely below-average. The reason is that the majority of pundits on our site are household names. In today’s prediction industry, a mainstream pundit is usually one who has perfected the playbook, making bombastic predictions simply to garner media attention. In other words, the mainstream pundit is all too often a bad pundit. This election prediction season was a perfect example: many of the political pundits we tracked are arguably not pundits at all but rather partisan mouthpieces (see: broken clocks). This is simply a function of the environment: we would love to track those who refuse to employ the playbook, but they aren’t on television.
This problem triggered the idea for the second twist to our website: providing users with a platform to make predictions. When you vote on the likelihood of a pundit’s prediction, you are effectively making a prediction of your own. This enables us to score our users the exact same way we grade pundits. By leveling the playing field between “pundits” and “users,” we can introduce a much-needed dose of meritocracy into the system.
By deflating inflated pundit reputations and surfacing the good pundits, we strive to disrupt the prediction industry. This won’t be an easy task, as there are plenty of entrenched interests at play and the system is wired for self-protection.
The good news is that our timing may prove fortuitous. “Moneyball” has popularized the notion of data-driven analysis, and given the public’s growing demand for accountability, we believe the stars are aligned for disruption. For this to happen it needs to be a collective effort. While we are ramping up the number of pundits tracked on PunditTracker (now more than 120), we readily admit that there are many more to catalog. So the next time you see a prediction on television, hear one on the radio, or read one on the Internet, take a second to send it our way to track and ultimately score. With your voice and our platform, we can finally bring accountability to the prediction industry.
We would like to close by thanking our users for all their suggestions for the website. Keep ’em coming!
We started tracking Doug Kass of the TheStreet.com this year (click here for his PunditTracker profile page), starting with his 15 surprises for 2012. Here is how Kass describes his surprises:
<< It is important to note that my surprises are not intended to be predictions but rather events that have a reasonable chance of occurring despite being at odds with the consensus….. In sports, betting my surprises would be called an “overlay,” a term commonly used when the odds on a proposition are in favor of the bettor rather than the house. The real purpose of this endeavor is a practical one — that is, to consider positioning a portion of my portfolio in accordance with outlier events, with the potential for large payoffs on small wagers/investments. >> [thestreet.com -- click here for his list of surprises as well]
As Kass notes, the standard “hit rate”/”batting average” metric is not suitable to evaluate his predictions. Instead, PunditTracker’s scoring system is ideal, given that our “$1 Yield” metric explicitly incorporates the boldness of predictions by measuring the average payout had you bet $1 on each of them, based on consensus odds at the time. A yield of exactly $1.00, for instance, means the pundit’s predictions were no better or worse than the consensus view at the time.
While there are still several weeks to go in the year, we figured it was an appropriate time to quickly review how Kass’s surprises are shaping up:
Sears Holdings declares bankruptcy (High Boldness)
- FALSE: 0/1; Running $1 Yield = $0.00
S&P 500 rises to at least 1527 (High Boldness)
- While the S&P is up 10% this year, it has failed to surpass 1475 (let alone 1527)
- FALSE: 0/2; Running $1 Yield = $0.00
Defcon Three Alert (High Boldness)
- FALSE: 0/3; Running $1 Yield = $0.00
One-third of current ETFs are forced to close (High Boldness)
- While the spirit of Kass’s prediction has come true (Morningstar has dubbed 2012 as the “Year of the ETF Closure“), with about 100 ETFs expected to close, Kass’s one-third figure is far too high.
- FALSE: 0/4; Running $1 Yield = $0.00
Israel attacks Iran (High Boldness)
- FALSE: 0/5; Running $1 Yield = $0.00
Bill Clinton and George Bush form a bipartisan coalition (Medium-High Boldness)
- FALSE: 0/6; Running $1 Yield = $0.00
Mitt Romney wins the election (Medium-High Boldness)
- FALSE: 0/7; Running $1 Yield = $0.00
Real GDP accelerates to over 3% in the second half of 2012 (Medium Boldness)
- Revised Q3 GDP growth was 2.7%, but consensus is for Q4 to be lower
- FALSE: 0/8; Running $1 Yield = $0.00
Buy European Stocks: MSCI Germany & MSCI France (Medium Boldness)
- MSCI Germany is currently up 23% YTD, while the MSCI France is up 19%. This compares to 13% for the S&P 500.
- TRUE: 1/9; Running $1 Yield = $0.22
Buy high-quality muni bonds (Medium Boldness)
- TRUE: 2/10; Running $1 Yield = $0.40
- Financial stocks are up 23% YTD versus 13% for the S&P
- TRUE: 3/11; Running $1 Yield = $0.55
- S&P High Beta index up only 5%
- TRUE: 4/12; Running $1 Yield = $0.67
- India ETFs are up around 27%, while China is up only 13%
- FALSE: 4/13, Running $1 Yield = $0.62
- While none have been acquired, they are up 28% on average
- TRUE: 5/14; Running Yield: $0.71
- Up 18% on average
- TRUE: 6/15; Running Yield: $0.80
Final Tally: 6 of 15 predictions correct; $1 Yield = $0.80
Interestingly, if Kass had just stuck to financial predictions, he would have been 6-for-11 with a very solid $1.09 yield.
We are now tracking more than 120 pundits on PunditTracker.com. With the 2013 Predictions rolling in, check out all the latest predictions and enter the PT Challenge to compete with the “experts” and win prizes.