The Wisdom of Crowds (James Surowiecki) - Book Summary

There are two distinct theories about crowds. One theory holds that crowds are "unconscious", chaotic, and blind. The opposite theory holds that crowds are "conscious" and have intelligence under certain conditions. The book  The Wisdom of Crowds  states that when the crowd is not rebellious or blinded by the group's own thinking, they will be smarter and more intelligent than any outstanding individual.

Author James Surowiecki specializes in business and finance for "The Financial Page" column for The New Yorker. His articles have appeared in many prestigious magazines such as  The New York Times, Wall Street Journal, The Motley Fool, Foreign Affairs, Artforum, Wired and Slate. He is a doctoral student in American History at Yale University.

Crowds predict the weight of a cow

In 1907, British scientist Francis Galton made an interesting discovery at the Cattle and Poultry Exhibition in the West of England. 800 people bought tickets to a prize contest: predicting the weight of a castrated bull regardless of its hair. There are many people who are not knowledgeable about cattle among the 800 participants in this competition. But surprisingly, the average valid result of the 787 players was 1,198 pounds, which is close to the actual weight of the cow after slaughter and hair removal of 1,197 pounds.

Witnessing this, Galton wrote:  “The results appear to enhance the reliability of the democratic judgment above expectations.”

Crowd wisdom

Following in Galton's footsteps, other researchers in various fields have proven the existence of the "Wisdom of the Crowds". Accordingly, when the crowd is neither rebellious nor blinded by the group's own thinking, they are smarter than any excellent individual at coming up with solutions to three main types of problems:

  1. Cognitive problems and problem solving. Crowds are good at solving problems with specific answers, such as: Who will win this year's Super Bowl? Where is the best place to build a factory? What is the probability that this drug will be approved by the US Food and Drug Administration(15)?
  2. Coordination problem. Crowds are pretty good at coordinating member activities. For example, how do buyers and sellers find each other and conduct a transaction at a reasonable price? How do companies organize their activities?
  3. Cooperation problem. Crowds are likely to cooperate. For example, how can self-interested and skeptical people work together and contribute to the common good? Paying taxes, dealing with pollution, and agreeing on a reasonable level of contribution are all matters of crowd cooperation.

Some examples, amazing evidence of the wisdom of crowds

In May 1968, the American submarine Scorpion went missing while on its way back to Newport from a mission in the North Atlantic.

The US Navy carried out this difficult search. Officer John Craven devised a way to find ships based on the wisdom of the crowd. He assembled a large group of knowledgeable people in many fields, including mathematicians, submarine experts and lifeguards.

He asked each of them individually to answer a series of questions about the Scorpion's disappearance, such as why the ship crashed, how fast it was traveling, at what speed it sank, and the like. .

Then, John Craven asked each person in the group to bet on their answers in order to exploit their thinking ability and knowledge about the accident. He aggregated each person's responses to draw an overall conclusion describing how the Scorpion went missing. Craven repeated this process to find a composite answer and locate the sunken ship. Later, the Navy found the Scorpion only about 40 meters from Craven's forecast.

At 11:38 a.m. on January 28, 1986, the space shuttle Challenger was launched into space. 74 minutes after reaching 10 miles and in flight, the ship exploded.

Soon, the stock market reacted negatively to the shares of the four contractors involved in the space shuttle project. It was Rockwell International building the ship and main engines, Lockheed managing the ground support, Martin Marietta building the external fuel tanks and Morton building the solid-fuel booster rocket .

Shares of these four companies have plummeted from 3% to 8%. The next day, shares of three of the four companies began to recover, down 3%, while Morton Thiokol's stock fell 12%. It was as if the market had found evidence to hold Morton Thiokol primarily responsible for the Challenger disaster.

Six months later, the inspection board officially concluded that it was the poor weather resilience of the O-rings, manufactured by Morton Thiokol, that caused the accident, and that the other companies were exempted. except liability.

Further investigations into whether there was a leak from Morton Thiokol's management were fruitless. (Prior to the results of the investigation, Morton Thiokol's management was not at all convinced that they were at fault. And if they were, they would have protected the information and not disclosed it to the market.) Apparently the crowd of investors. It was clever in the stock market to find out exactly which company was at fault just 30 minutes after the accident.

Google's exact search engine also relies on the wisdom of the crowd. Yahoo, Alvista and Lycos were the leading search engines until Google appeared in 1998. At the core of this search engine was the Page Rank mechanism invented by Sergey Brin and Larry Page in 1996.

This is a calculation method that allows all websites on the Internet to decide which pages are most relevant to the information being searched, specifically as follows: The Page Rank algorithm considers the link from page A to page B as one vote from site A to site B. Google will rate and consider the importance of the pages. By aggregating the number of votes, the links and the importance of the votes, the Page Rank algorithm will produce results - ie pages related to the search content - with a corresponding level from high to low. .

When does the crowd have wisdom?

The crowd is not always wiser than the individual, especially the best. There are four conditions necessary for an intellectual crowd, including:

  1. Diversity of opinions. Each person should have some particular piece of information, even though it may be a particular interpretation of known facts;
  2. Independence. People's opinions are not formed according to the opinions of those around them;
  3. Decentralization – decentralization. There is no direction (from the center) for each team member;

4.  Synthesis. The group must have a mechanism to turn individual opinions into collective opinions.

Diversity – the story of the 19th century American auto industry and how bees make honey

The early period of the American auto industry, as well as other industries such as railways, television, personal computers, e-commerce, etc., all had one thing in common: there were many companies, many choices and many different models.

Over time, the market will gradually distinguish winners and losers. Ineffective models and technologies are eliminated, leaving only effective models and technologies. As a result, many companies fail and society wastes billions of dollars on models and technologies with no way out. This is not a wise choice. Consider how the swarm of bees distributes honey.

The bees do not discuss where to search, instead they send many bees to fly in many directions to search for honey in an area 2 to 6 km away from the hive. The bee that finds a source of honey will return to the hive and perform a tail wagging dance to announce the news. The intensity of this tail wagging is directly proportional to the abundance of bile they have obtained. As a result, the entire colony of bees will focus on the place with the most honey source without being dispersed to many places. The bee selection process consists of two simple stages:

  • Stage 1: diversification, i.e. more bees looking for possible possibilities;
  • Stage 2: decide to choose one of those possibilities for the whole team.

This is different from the actions of companies in the early days of the industry. They have spread out to set up companies according to many models and technologies, but do not immediately decide on an effective technology to work together.

Crowd diversity is very important. Studies show that a random group of non-experts is more effective than a group of experts when it comes to problem solving. Similar knowledge and experience of experts undermine the effectiveness of diversity. Meanwhile, the different combined knowledge and experiences of the members of a diverse team will be wiser and smarter than even the best experts. Another important point is that the diversity of the group will lead to the independence of opinions from the group members.

Crowds need independence of ideas to be intellectual

The independence of ideas of team members is important for smart decision making for two reasons.

  • First, it doesn't let people's mistakes correlate to lead to misdirections.
  • Second, members with different abilities will have newer information from other perspectives, rather than old data from points of view that are already familiar to everyone.

Independence is very important to the wisdom of the crowd. However, this is difficult to achieve because people tend to influence each other. When faced with great influence, people often behave in a herd manner. The source of this behavior is that people tend to communicate information and act on “social proof” – that is, what crowds do.

For example, when we go through an intersection, if we see most of the people there looking up at the sky, then there is a high chance that we are also looking up at the sky. When we see many people queuing at a restaurant, we often think that the restaurant has good cooking or something special.

It is this crowd behavior (herd mentality) that has created stock bubbles in world markets and many other unpleasant phenomena. However, if we know how to "take advantage" of the psychology of the crowd, we will do our job well.

For example, to encourage residents in a Los Angeles suburb to recycle, Wess Schultz divided residents into two zones. One area received a leaflet outlining the reasons for recycling, the benefits of recycling, and how to do it; and the other received leaflets stating how many residents in other regions did the recycling and how they did it. The second way – using the crowd effect – worked.

Need for decentralization – decentralization helps the crowd retain diversity and independence

Decentralization helps the crowd retain diversity and independence – which are essential for crowds to stay smart. However, decentralization or decentralization also matters, that is, important information is not shared among members. To solve this problem, the crowd must continue to be consistent in decentralization and build a system of linking and sharing important information among members.

The Future MAP project is proposed to collect information and opinions of the people through betting on possible terrorist events. This information will be shared with members of the analysis team. This project was strongly opposed by the US Senate and many people, but this is really a very good plan to collect information from the crowd.

Crowd cooperation

Crowds are more cooperative than we can imagine. In 1996, Whyte, author of  The Organizer (16), observed pedestrians moving through the streets of New York and discovered the subtle cooperation of the crowd. Although there are a very large number of people and each person has a different way of moving, they still coordinate smoothly to give each other space, avoid collisions, and avoid being hit by cars at intersections; and this huge crowd is constantly operating on the streets of New York.

In 1958, social scientist Thomas Schelling conducted an experiment with a group of law students from New Haven, Connecticut. The exercise for each person was that on that day, they had to pick up someone in New York City but didn't know who to pick up, didn't know what time to pick up and didn't get in touch with them in advance. The results were quite surprising: the majority of students chose to pick up at the Information Room of the City Center Station, and at midday.

In another experiment, he showed a crowd a box divided into 16 squares. Each person ticks a box of the box. They will be rewarded if everyone ticks this square. The result is that 60% of players have ticked the top left square.

As Schelling explains, there is a focal point, or common meeting place, in everyone's head. This point is called the “Schelling” point. The existence of this "Schelling" commonality proves that the experiences of people who are very different, are also very similar, and that they can cooperate without common direction or information for each other. together.