In God We Trust, all others bring data

The 21st century is built on the cult of the statistician, who by their skills at analysing, managing and explaining (diving, conjuring) meaning in complex human systems, tells us new truths about ourselves that we would be blind to without the data. Statistics have become popular in all sorts of ways: in books such as 'Freakonomics' and 'SuperFreakonomics', by Steven Levitt and Stephen J. Dubner; the baseball analyst Paul DePodesta who was the subject of the film 'Moneyball'; and the elections analyst Nate Silver. 

The statistician is wise and all-knowing, another facet of attempts to apply the aesthetics of scientific reasoning to everyday life (see the upswing in hip Atheism, love of geeky shows like 'The Big Bang Theory', and geek-power texts such as 'The Geek Manifesto': all part of a revival of the 50s "gee-whizz, ain't science wonderful" an approach further consolidating the technocrats as appropriate holders of post-political power). The 2012 American elections seem to have been more about the polling data and Nate Silver's predictions than about the candidates themselves. Some Republicans seem to think they lost the race to Silver's data, not Obama.

What this infatuation with the aesthetics of science (its procedures, its sense of purpose, and its look) does not acknowledge is the fusion of everyday life with science was tried before many times in the 20th century: reshaping human society for the better, through purely scientific means, largely failed because by applying scientific reasoning, the systems that scientists, economists, and politicians created were separate from the lives of the people and ignored existing power systems. 

Correlation Does Not Imply Causation

William Edwards Deming was born in 1900.  His family moved to Wyoming in 1907 to settle on land which turned out to be useless for farming; conditions were poor and they were often in debt. The Demings were well educated however and their son William attended university in Wymoing, Colorado, and later Yale: he received graduate degrees in electrical engineering, mathematics and mathematical physics. He was also keen on music composition and wrote a simplified version of the Star-Spangled Banner because he thought the original was too complex a composition.

Deming worked at the Department of Agriculture, and met Walter A. Shewhart, the inventor of statistical methods of quality control. Shewhart worked in the engineering department at Bell Telephones: in 1924 Shewhart showed his boss a piece of paper. Shewhart's paper was a control chart, and showed how the production of equipment could be rationalised at Bell: instead of checking finished products for faults, Bell should use a process of statistical control in its manufacture, so mistakes would only ever occur by chance and future outcomes could be predicted. Deming learned all of this from Shewhart and started applying it not just to industries as Shewhart had, but to the management itself.

Deming had also met another proponent of rationalization through science, the British statistician and biologist Sir Ronald Fisher. Fisher was a main figure in the Neo-Darwinist, or Synthesis, movement in evolutionary theory. Fisher applied his theories about evolution in a statistical way: amongst the statistics based theories he presented in 'The Genetical Theory of Natural Selection' was the idea that attributed the rise and fall of civilizations to the fertility of the upper class. Fisher was right, he said, because the 1911 census data, containing information on fertility, proved it. Fisher was also in favour and was a supporting member of the Eugenics movement which tried to pass sterilization laws in the 1930s. When Richard Doll and Austin Bradford Hill published their research in 1956 showing a link between lung cancer and smoking, Fisher used the same methods to show a correlation between divorce rates and the import of apples.

Field Experiments

The Americans in 1945 had won a victory over Japan not just in military terms, but in cultural and scientific ones as well. A policy of technocratic military planning (cost-benefit analyses of which islands were worth taking and which were worth leaving alone), and of a willingness to use atomic weaponry to force a game theory resolution, had been used to win the conflict. The Americans, led by General Douglas MacArthur, moved into Japan determined to reshape the country. MacArthur's title was the Supreme Commander of the Allied Powers, and he held enormous power in Japan during the occupation from 1945 to 1951. MacArthur increased censorship in the Japanese media making subjects such as the discussion of the atomic bomb illegal; he granted immunity to war criminals such as the physicians in Unit 731, who had conducted cruel and deadly biological and chemical experiments on the Chinese population. As with Nazi scientists who engaged in war crimes, they were offered protection because, as MacArthur put it in a letter to Washington in 1947: "additional data, possibly some statements from (Shiro, Unit 731 commander) Ishii probably can be obtained by informing Japanese involved that information will be retained in intelligence channels and will not be employed as 'War Crimes' evidence."

MacArthur also reformed Japanese business, and his main enemies were the zaibatsus, family-controlled monopolies which had dominated Japanese industry. The zaibatsus had done well during the run-up to the Second World War but afterwards they were not looked on in favour by the Americans: the Roosevelt administration had attacked monopolies in America and staffed its departments with young technocrats willing to do the same around the world. MacArthur dismantled many of the zaibatsus, but the Americans never completed the project of complete dissolution: Mitsubishi and Nissan are two of the most notable who have survived.

If Japan Can, Why Can't We?

In 1947 Deming went to Japan to help the occupying forces create a census of the Japanese population. He came to know some of the new businessmen and managers leading the companies that had come into existence to fill the gap left by the zaibatsus. In 1950 Deming began lecturing Japanese industrialists on the statistical methods he had developed from Walter Shewhart's ideas at Bell. Although the managers of the companies themselves did not believe him initially, Deming said that by adopting these methods Japan would become a successful and powerful country: Deming said it would only take them five years. What Deming taught the managers was this: ensure good quality design, standardise production, improve testing of products, and sell them globally. Here is a copy of the only known translation of Deming's lecture in 1950. Deming hated management, and regarded it as the biggest obstacle in the way of success.

The businessmen and managers Deming talked to went on to apply his ideas: Japan very quickly began to rise in the industrial stakes. Japan had become one of the world's economic superpowers by the 1980s as a result of several factors: the American efforts to rebuild the economy postwar; close government relations with (largely anti-communist) trade unions; and the Ministry of International Trade and Indsutry, an organization with the power to co-ordinate Japan's industrial efforts. A pivotal figure in Japan's growth was a former head of the MITI, Prime Minister Hayato Ikeda (1960-1964): Ikeda increased the pace of economic development at breakneck speed, and was forced to resign from his role as head of the MITI in 1952 when he said "it makes no difference to me if five or ten small businessmen are forced to commit suicide" due to the pressures of the reforms.

There was widespread fear in the '70s and '80s in America that Japan was going to drive America out of business with its flood of cheap, high-quality imports. This fear was expressed in a number of ways: anime for example became popular, there were nods to future Japanese dominance in films like Die Hard (Nakatomi Plaze). The sociologist Ezra Vogel wrote a bestseller in both America and Japan called 'Japan as Number One', which gained popularity for explaining in simple terms why Japan had outpaced the west and why the Japanese enjoyed benefits unheard of in the west, such as low crime.

The Americans started to investigate why the Japanese had come to dominate areas such as the electronics and automobile markets: to their surprise they found an American partially responsible. Deming was invited in 1980 by NBC to appear on a news report called "If Japan Can... Why can't we?" Deming explained his methods to the audience, and railed against management as the cause of all evils in the workplace. American businessmen, who had paid no attention to Deming beforehand, began adopting his methods; he became extremely popular and acquired a cult reputation. Deming went on to work for Ford, before dying 1993.

Meanwhile in Japan

During the occupation of Japan, the Americans turned their attention on Japan's organized crime networks, the Yakuza. Like the zaibatsus they saw the yakuza as a threat to Japan's stability, a traditionalist inefficient hangover from the past. The yakuza had control of the country's black market, and leveraged political and economic power out of the Americans attacks on Japanese business which had left space for them to occupy. They also helped the Japanese state to crush labour unions and leftists: Yakuza leaders such as Yoshio Kodama, and Hisayuki Machii, aided right-wing political parties and had a relationship with American intelligence. They were used to attack striking workers as well. Under Kodama, the yakuza ceased warring with one another and became intricately involved with the Japanese, and eventually American, markets: Kodama was behind the Lockheed bribery scandal of 1976, the first incident to show the extent and power of the yakuza, which had a massive impact both in Japan and America. Kodama narrowly escaped an assassination attempt by a pornographic actor, who crashed his plane into Kodama's house. The yakuza became deeply involved in Japanese politics: the ghosts of the traditional forces that Deming had tried to banish with scientific methods would not stay silenced. In 2012 Keishu Tanaka, Japan's Justice Minister, resigned over allegations of connections with one of the biggest Yakuza organizations, showing the extent to which traditionalists had entrenched themselves in the political order.

Japan's rise in the postwar period had been meteoric: not only had te economy boomed, Japan had also appeared to have humiliated its old war enemy America, using the very methods and infrastructures they had left behind from the occupation. Confidence was very high. The real estate market in Japan boomed; there was also stock speculation. For a time it cost $2 million for an average house outside Tokyo. In 1990, the market crashed.  Japanese companies began laying off their workforces, and the companies that had profited after the occupation began losing out to competitors in South Korea and Taiwan. The 1990s became known as 'The Lost Decade', and the economy has been unable to recover since. The problem the Japanese encountered parallels the situation with the world economy now: in 2009 President Obama compared the years of stagnation in Japan with the crisis America is facing now: no growth.

There is a new superpower in the east, perceived as threatening the west's power: it is China. The Chinese economy seems unstoppable but already, like Japan in the 1980s, there are indications that the housing bubble there is about to explode. The question is, with the American and Chinese economies so intimately linked, what happens if or when it does?