Posted on May 5, 2008
Filed Under BS-Free Advice |
In recent posts ( and here) I’ve made it clear that I apply the “trust no one” mantra to many things, including research firms and well-connected individuals.
This, of course, is not to say that I don’t listen to what “experts” have to say. “Expert opinions” can be extremely valuable when used appropriately.
Unfortunately, some “experts” aren’t “experts” and implicitly trusting that every statement or prediction that comes from somebody you perceive as being an “expert” is accurate or wise is not the best strategy.
I try as much as possible to rely on common sense and logic to evaluate situations and even if the results prove my critical thinking to be wrong, I much prefer to make bad decisions on my own as opposed to making bad decisions based on third-party information I accepted blindly.
Another amusing example of just how foolish it can be to rely on “experts” was provided by Citi Internet Analyst Mark Mahaney. On February 2, the day after Microsoft announced its bid for Yahoo, that there was a 60% chance that Microsoft would wind up acquiring Yahoo, noting a 20% probability that the original bid would be accepted and a 40% probability that a higher bid would be accepted.
On April 25, he upped the total probability that Microsoft would acquire Yahoo to 90%.
On April 28, in a TechCrunch interview with Mahaney that Mahaney did not think Microsoft was likely to simply walk away.
The interview, of course, demonstrates that Mahaney, as to be expected, based his predictions on certain assumptions and the best knowledge he had at the time. He appropriately admits when he is unable to answer a question, at two points stating “Sorry Michael, I don’t have a good opinion on that” and “That’s a good question, Erick. I don’t have a well-formed opinion on that.”
Fast forward to today.
Microsoft has walked away and although there remains the possibility that the move will eventually lead to a scenario in which Microsoft and Yahoo get together, Mahaney now says a Microsoft-Yahoo deal is “still 15% likely” the TechCrunch headline.
If that isn’t headline spin, I don’t know what is.
Of course, the purpose of this post is not to criticize Mahaney, who may or may not be worth listening to (I’m not familiar enough with him to express an educated opinion). The purpose of this post is to reiterate the same point I made about research firms:
To be fair, research firms like Forrester are tasked with an almost impossible task - predict the future. Nobody can. At best, educated speculation can be provided.
Even then, significant differences are bound to exist, highlighting the fact that even educated speculation is still speculation.
Predicting the future is not possible, and the vast majority of the time, numeric probabilities coming from non-mathematical analyses should be avoided like the plague. After all, they try to put a quantitative spin on a qualitative evaluation - something that just doesn’t work. Only numbers derived from a fundamentally-sound mathematical analysis should be (potentially) taken seriously.
While I always chuckle when guys like Mahaney give such probabilities and then are forced to change them dramatically as is the case here, the real problem, of course, is that many people actually give significant weight to these numbers and consider them credible because they trust the expertise of the person who came up with them.
That’s precisely why research firms and analysts create them in the first place - people want a simple piece of information that requires little thinking to digest.
This is highlighted by Michael Arrington’s rationale for interviewing Mahaney:
Given how consistently right Mahaney has been in his analysis, Erick and I reached out to him today to pick his brain on the deal.
And is further demonstrated by Arrington’s statement that, after being told by Mahaney that (if you’re Ballmer) “You can’t walk away”:
There are very few people who understand this situation as well as Mahaney. If he says Microsoft needs Yahoo, I’m not going to disagree.
This is emblematic of the challenge that exists when it comes to research firms and analysts. As I stated:
The big problem, of course, is that research firms are often looked upon as authoritative sources. Many investors are easily seduced by the projections that are produced by research firms and if a research firm predicts that the podcasting market will be worth $1 trillion by 2012, there are some individuals who can’t be convinced otherwise.
Implicit, unquestioning trust is never good and reliance on the belief that the past predictions of a particular firm or individual have been good ignores a simple, yet wise, adage: “Past performance is not necessarily indicative of future results.”
Had you made trading decisions on MSFT and YHOO, for instance, based largely on Mahaney’s predictions, the fact that you found his past predictions to have been “solid” would probably not provide much comfort right now given your potential losses - despite the glimmer of hope that there’s “still” a 15% chance of a deal.
Perhaps that most important take-away is that, even if you have established that a person or business has “expertise,” the type of information that is most appealing, such as predictions (i.e. “there is a 90% chance that Yahoo will be bought by Microsoft” or “the podcasting market will be worth $50 billion by 2012″), is exactly the type of information that is still not very reliable no matter who it is coming from.
In general, I’ve found real experts to be capable of predicting general trends quite well but generally incapable of reliably predicting the magnitudes of those trends and the timeframes in which they will be realized. And when it comes to the outcomes of specific short-term events, in particular, I will say little more than caveat emptor.
Print This Post