On Statistical Distributions: A Review of "The Black Swan"

A few weeks ago, the Wall Street Journal published a review of the book, "The Black Swan," written by Nassim Taleb, and reviewed by David Shaywitz.

Taleb is currently a hedge fund manager, and, as such, takes particular notice of the distributions of various types of events. Distributions are important....they can undermine models when inappropriately chosen. Parenthetically, this Friday's Journal carried a piece on the recent cheating scandal at Duke University's Fuqua Business school, and I could not help wondering how much such cheating results in even more mediocre, mis-educated people wind up in positions where they can do real damage to others.

From other reading about Taleb, I know that he is keenly sensitive to a specific sort of anomaly- that of sudden and unpredicted market swings of the type experienced on the heels of Long Term Capital Management's difficulties in 1998. The following passage is from a 2002 New Yorker article about Taleb,

"Physical events, whether death rates or poker games, are the predictable function of a limited and stable set of factors, and tend to follow what statisticians call a "normal distribution," a bell curve. But do the ups and downs of the market follow a bell curve? The economist Eugene Fama once studied stock prices and pointed out that if they followed a normal distribution you'd expect a really big jump, what he specified as a movement five standard deviations from the mean, once every seven thousand years. In fact, jumps of that magnitude happen in the stock market every three or four years, because investors don't behave with any kind of statistical orderliness. They change their mind. They do stupid things. They copy each other. They panic. Fama concluded that if you charted the ups and downs of the stock market the graph would have a "fat tail,"meaning that at the upper and lower ends of the distribution there would be many more outlying events than statisticians used to modelling the physical world would have imagined."

In effect, the behaviors of people which follow power-law distributions can screw up financial theories and their implementations. e.g., LTCM, when those theories and their implementations are based upon the wrong distributional assumptions.

The larger lesson from Taleb, beginning with Fama's observation, is that purposeful, goal-directed human behavior is probably not random. Thus, it should not be assumed to approximate a normal distribution of possible occurrences.

Personally, I am not at all surprised by this phenomenon of mistaking one distributional assumption for another. It's been my experience, and belief, since 1985, that the inclusion of powerful statistical tools in spreadsheet programs began the inexorable slide in the quality of statistical analytical applications in business. In that year, a University of Chicago-educated MBA colleague of mine at the Chase Manhattan Bank managed to mangle a rather straightforward piece of stock price movement attribution analysis, thanks to his access to a regression analysis tool in Lotus 123, and an ignorance of how to build valid models.

This example of how a little knowledge can be a dangerous thing has, I suspect, been multiplied many times since in portfolio management and trading operations the world over.

Taleb's key insight is that what are often seen as 'once in a lifetime' events which are unrelated, per se, are, in fact, neither 'once in a lifetime, nor, strictly speaking, unrelated. To quote Gilda Radner's SNL character, Roseanne Roseannadanna,

"....it's always something......"

It certainly is.

1987: a major market drop of proportions not seen in one day since 1929.

1989: Chase Manhattan Bank's failure lend to UAL's ESOP for a buyout triggers this 'little crash.'

1995: Barings Bank fails due to Nick Leeson's trading losses and fraud in concealing them.

1997: Asian financial markets crash.

1998: LTCM's incorrect assumptions regarding various hedged financial instruments' behaviors under stress results in the firm's demise, and a brief but severe market downturn.

2001: 9/11 attack on World Trade Center towers causes a market slide already in progress for a year to deepen and lengthen.

2006: Amaranth hedge fund fails from overly-risky, wrong bet by one trader on natural gas prices.

Thus, courtesy of Fama and Taleb, we see that there is considerable risk in assuming that severe market disturbances of the sort which occurred in 1929 do not recur very frequently. And a market composed of investors and traders who predominantly model strategies with risk metrics that employ the wrong, normally-distributed assumptions about the frequency of occurrences of various outcomes, is inevitably going to react badly to the actually more-frequent occurrences of extreme outcomes which affect financial markets.

Perhaps my own approach, using consistency of membership in a class, rather than forecasting the value of each member of the class, is less susceptible to this type of risk.

However, it goes even further. Thanks to reading this review of Taleb's book, and the 2002 New Yorker article, my partner and I have developed a powerful new variant of my equity strategy which explicitly takes into account Taleb's preoccupation with extreme market occurrences.

I'll write more about this in an upcoming post.