Long Tail Distributions
A long tail distribution is one that has very few occurrences of very large events, and very many occurrences of very small events, which gives the graph a “long tail.” This “long tail” is generated by a power law relationship, meaning there is a power relationship between the size of an event and the likelihood of its occurrence.1 In statistics, a long tail of some distribution of numbers is the portion of the distribution having a large number of occurrences far from the “head” or central part of the distribution. They are typical of nonlinear systems, as nonlinear systems are “non-normal”, meaning there is statistically no normal average or typical state within the system. With linear systems, there is some kind of mean state to the system. Given enough samples of the different states within the system, we will be able to compute some average that we can use as a representative of the whole system. For example, the distribution of people’s height, if we were to plot them, would follow what is called a normal distribution, meaning there will be very many people around the average of say 5 to 6 feet, some a bit larger and some a bit smaller than this, say between 3 and 8 feet, but virtually no one outside of this range of states. These normal distributions then have a well-defined center and then drop off exponentially fast, meaning there is an extraordinarily low probability of getting extreme states.
A feature to normal distributions is the so-called law of large numbers, which means that the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed. This so-called law is important because it “guarantees” stable long-term results for the averages of some random events. For example, while a casino may lose money in a single spin of the roulette wheel, its earnings will tend towards a predictable percentage over a large number of spins. Any winning streak by a player will eventually be overcome by the law of large numbers that will bring the net gains and losses back to an average.
The normal distribution holds for many linear systems, physical and chemical, but in the world of non-linearity, the idea that there is such a thing as normal and we should expect this normal is no longer applicable. The power law nature of nonlinear systems creates what is called a long tail distribution, meaning extraordinary events that are virtually impossible within normal distributions are possible within nonlinear systems. These extraordinary events are referred to as “black swans”. For example on October 19th, 1987 on Black Monday, the Dow Jones stock market index dropped by 22% in a single day. Compared to the typical fluctuation of less than 1% this was a shift in more than 20 standard deviations from the norm. Within a normal distribution, this would be virtually impossible, with something like a 10 to the power of 50 chance of it happening.
These extraordinary events can then have a dramatic effect on the system’s average behavior because if we take a random sample we will get a certain average but then if we add just one more node to this it might be a black swan that will radically alter the average again. In the example of measuring people’s heights, if we have a room of people and then the tallest person in the world walks in, the average height will only change by a few feet. But say we are measuring people’s income and now the richest person in the world walked in with an income of many millions, this would so radically alter the average income for it to become nonsensical. In a power law distribution as we increase the number of samples we take, values will not converge to an average. They will, in fact, diverge, with some exceptions. Asking for an average is like asking how big is a stone or how long is an average piece of string?
The normal distribution is largely derived from the fact that we are taking random samples from components that have no correlation between them. If I flip a coin now, it will not affect what I get on the next flip and will thus follow a normal distribution. If one person wins on one casino table, it will not affect whether someone else will win on another and so on. But in nonlinear systems, things are arranged in a particular way. Large websites are large because of the network effect and because people have specifically chosen to connect to them. There is nothing random about this. Financial crashes are also similar in nature.