An old German 10 Deutsche mark bank note featuring Gauss and his bell curve

I was talking to Nassim Nicholas Taleb about Fooled by Randomness, which I’d just read, and his forthcoming (2006) book Black Swan.

He was fuming about an old ten-mark bank note.

“Money is the last place to put a bell curve! A bell curve has nothing to do with money — it’s a power law.”

The virtues of the long tail on the Internet are memorialized daily, but it’s well to understand that the long tail doesn’t just mean new customers for smart marketers. The “long tail” just means things that don’t occur very often. For instance, people who search very specific strings of rare keywords; the election of an American president with brains and character; or a massive spike in readership of this blog.

I sat with Nassim and my friend Peter Ayton. They had both just come from the Society of Judgment and Decision Making conference in Toronto. Normally dominated by geeky decision-making scientists, Nassim was invited as a keynote speaker (PDF) where he gave a talk called The Black Swan: Why Are We Still Blind to It, about rare events and why no-one takes them into account. In his view, while extreme “black swan” events are rare, they follow power law distributions (also known as Pareto’s law or Zipf’s law), just like the famous long tail — and they should be expected, rarely. When they do occur, they can have enormous consequences.

Nassim listened to my story of having created and sold a few companies. Normally this well-rehearsed litany inspires polite congratulations at a minimum, but he was unimpressed. I later understood that he thought it entirely possible that I was just lucky.

Nassim’s strong claim is that in economic matters, Gaussian distribution curves (like the familiar bell curve) just don’t hold. The height of people in a crowded restaurant follows a bell curve — it’s wildly unlikely that you would see a 20-foot man, and no-one, no matter how tall or how short, would change much the average height of the restaurant’s patrons.

But if Bill Gates walked into an East Village coffee shop, he would be, in economic terms, a 3,000-foot man. He would skew the mean net worth of people in the room very substantially. His billions would give the crowd an average fortune of millions, even if Bill was the only one there who could afford to pay the rent. That’s because income distribution follows a power law. Most of the wealth in the world is held by a tiny percentage of its population. The rest of us are squirming around in the long tail of a power curve.

In Nassim’s view, anyone who sets the limits of the possible in economics by using distributions taken from biology is seriously mistaken. And yet we do it all the time, quite naturally. It’s what we’re used to.

Nassim delights in the come-uppance of those who preach that they can predict the future, who use their past successes to convince people that they will win again.

“My major hobby is teasing people who take themselves & the quality of their knowledge too seriously & those who don’t have the guts to sometimes say: I don’t know….”

The fund manager, for instance, who plays the peacock because she’s had great gains for the past five years, then “blows up.” If you have enough fund managers, chances are that a few of them, without any skills or qualifications whatever, will do very well. If you throw lots of darts, a few of them will stick in the bulls-eye, even if you do it blind. Malcolm Gladwell writes more about this in Blowing Up, an entertaining article about Nassim’s career on Wall Street second-guessing financial analysts and watching them reach the end of their luck — and their careers.

The long tail of the power curve is not just a nice placid place where Billy the lonely enthusiast finds validation by joining an online Warhammer community, or where you can finally get those wonderful California chocolate-covered cherries online.

These distributions are rare enough, but not extremely rare. They are in fact common enough for people to build small businesses around, or perhaps even a big one if taken in aggregate. That’s the long tail that Internet pundits go on and on about, but they’re really talking about the fat part of the long tail.

At the very end of the long tail are extremely rare events, events that have never occured before and are therefore judged to be “impossible.” Before the Dutch brought back word of Australia, Europeans had seen only white swans, and deemed it “impossible” that there should be black swans until cygnus atratus was discovered 1697.

Weeks later, in a cafe full of bell-curve distributed business lunchers, I gave Nassim a map of meteor landfalls in North America. On the map, they appeared in clusters, as if in a pattern, as if intentionally. I thought this would be a nice gift because it illustrated how we can interpret random events as meaningful. Instead, he was politely exasperated: “Cancer clusters are a much better example,” he said.

Nassim characterizes history as a succession of “black swan” events upsetting established orthodoxies — randomness producing the “impossible.” Without randomness, structures of power would simply grow and become more and more unstoppable. Microsoft would continue to increase its hegemony; the United States, as the world’s only superpower, would expand its domination even more; hell, the U.S. would never even have existed, because it could never have defeated the gargantuan British Empire. In fact, we’d all be citizens of the Roman Empire.

We don’t truly appreciate the role of randomness; our ways of constructing meaning requires that we create narratives to explain events, and a story hates chance happenings. Our penchant for stories, our inability to understand the role of randomness, is a serious impediment to understanding reality. And our application of biological distribution curves to economics is a serious impediment to understanding business trends.

——————————————————————————–
Update March 6: Micromotives reports on a review of Fooled by Randomness by Columbia’s Andrew Gelman, as well as his study, which claims that statistical forecasting models can help estimate the probability of events so rare that they’ve never occured. Their test case: what’s the probability that your vote will decide the next U.S. election.

Tags: Nassim Nicholas Taleb, Peter Ayton, Malcolm Gladwell, Long Tail, power law, Paredo’s Law, Zipf’s Law, randomness, Gauss, bell curve, black swan, Fooled by Randomness

Posted by Antony in: General, Media + Marketing, Search