Sunday, October 25, 2009

More Bad Research On Corporate Performance From Deloitte

I read a reference in a recent Wall Street Journal column by Holman Jenkins to a recently-popular piece of research by two Deloitte consultants, Michael Raynor and Mumtaz Ahmed, along with a "researcher" from the University of Texas, Andrew Henderson.

The book they wrote is described here, and the paper from which it sprang, may be found here in a Deloitte website post, and downloaded/read.

Whenever I read of a reference to some new empirically-based work purporting to diagnose the bases of corporate performance that is desirable to emulate or duplicate, I naturally am curious to learn who did the work, the methodology employed, and their conclusions.

In this case, it appears that Raynor is the leader of the group. His bio can easily be found, as well as his own website. From what I've gathered, he is clearly an intelligent individual, but doesn't seem to be described as having any significant working experience in business, outside of his academic pursuits at Harvard (DBA) and as a consultant at Deloitte. Deloitte, it should be noted, is not exactly in the vanguard of management consulting. Neither is Harvard known as a source for the best empirical approaches to business performance analysis. And it would appear that Raynor is in a sort of 'of counsel' role at Deloitte, as he has his own speaker's bureau representative and website.

In the beginning of their article, the authors state that they believe most prior studies of excellent US businesses have, in fact, been portraits of lucky, rather than skillful firms.


It's also not all that surprising, to me, at least, to learn that Raynor's methodology has little relationship to the real world in which most businesses operate, i.e., a need to produce results that create wealth for business owners. The most easily-accessible data for this, which, conveniently, also is the business form which accounts for the bulk of US business activity, is total returns of publicly-held corporations.


Because I linked to the authors' original article, I won't duplicate their text with reposted passages here.


In their piece, the authors essentially put down total returns as too reflective of future performance, as anticipated by investors, than actual management skill.

Instead, they chose ROA as their preferred measure.

As I mentioned to a colleague, this choice, alone, virtually guarantees the uselessness of all of their efforts.

Yes, they got their article in HBR, won a prize, expanded it to a book. Fine. Tom Peters got a lot of accolades, too, at first. But his work sunk like a stone into the vast sea of strategy and management 'how to' tomes. As, I would expect, will this latest effort by Raynor, Ahmed and Henderson.

Their description of what total returns are is wrong. It's not simply a measure of future "surprises" to investors. Taken as a pattern, over time, total return measures, in a presumably reasonably efficient market of investors and analysts, the ability of a firm's performance to exceed expectations. It does involve expectations, but it also involves expectations based upon prior and evolving performance.

And, more importantly, it is the measure of wealth created by management of the firm, regardless of the exact source of that wealth. It may have been a fortuitous purchase of a patent, a discovery in the research labs, a marketing edge, the discovery of some mineral deposit, or other unpredictable competitive advantage.

In fact, the very unpredictability of the advantage is what generates surprises and wealth. If all gains or excellent performance stemmed from reproducible methods, then those methods would quickly be copied, implemented, and all competitive advantage due to them would vanish.

Such performance can't, won't, and doesn't typically last for very long. But that timeframe can be years, not days or months.

In fact, my own research began with my simple quest to learn what the distribution of company performances was on the basis of being able to consistently outperform the equity markets averages on total return. From that knowledge, I was able to discern a range which constitutes an average length of time of outperformance.
In the same research, I tested ROA's association with patterns and levels of market outperformance, and found it absent for firms which grew revenues at above-average rates. Simply put, ROA is a point estimate of little value in deducing ongoing behavior of firms that are growing at a healthy pace.


And ROA doesn't automatically or tautologically translate into shareholder wealth. So it's going to be of passing interest to both investors and CEOs.

Thus, for all their extensive, hard quantitative work, the authors of the Deloitte study have pretty much doomed it to insignificance because it doesn't generate operable conclusions which directly lead to increased wealth for shareholders or their CEOs.

Then there's the matter of choice of patterns of outperformance.

In my research, I first reviewed real corporate performance over time. From these analyses, I constructed patterning variables which grouped companies by the pattern which their performance exhibited.

By contrast, the Deloitte study authors began by arbitrarily deciding to set a threshold of occurrence of 9 out of 10 years for a variety of unspecified fundamental performance measures.

Why should 9 out of 10 be the appropriate screening value? Why impose a value on the date a priori, instead of simply letting the data describe the true situation?

Thus, the authors proceeded down a path which features their own subjectively-chosen patterns for outperformance on a measure, ROA, which has no direct relationship to the growth of shareholder value in most companies.

As I read their paper, I reflected on my own background being a curious confluence of several important streams of influence. Over many years and with different companies, as a marketing and strategy professional, internal consultant, external consultant and research director, in several different sectors, including financial services, I happened to absorb several key lessons for this type of research.

Being in consulting at Oliver, Wyman & Co., I didn't approach research on the sources of consistently superior shareholder wealth creation with the scepticism that a true believer in efficient financial markets. When I produced my financial services sector results and presented them to retiring Chairman Alex Oliver, he exclaimed, to paraphrase,

'For years we've been saying we had knowledge of what drives superior performance, but we never really did. Now, with this, we do. This is a strategy consultant's ultimate tool.'

Alex was a lot of things, including cheap and petty, but he was arguably one of the best strategy consultants of his time. In his day, he headed Booz Allen Hamilton's strategy practice, leaving to co-found Oliver, Wyman. I took his praise as justifiable proof that my research approach was unique, effective and applicable in a very pragmatic manner.

When I extended the research, on my own, to the entire S&P 500, the results were even more powerful and applicable.

What Raynor, Ahmed and Henderson have produced has no real applicability other than a sort of minor confirmation that what can be easily duplicated is of little lasting value. And that what typically creates significant shareholder wealth can't be reduced to easily-duplicated management dicta.

So, there you have it. The Deloitte study authors and I agree that unexpected innovation can't be easily duplicated as a management style.

But I already knew that, and, if you read this blog regularly, so did you.

No comments: