Bigger groups make better decisions

In his seminal work in 1907, Vox Populi (The Wisdom of the Crowds), Sir Francis Galton looked at the results of a weight-judging competition where participants had to guess the weight of an ox for a monetary prize. As would be expected, there were a wide range of guesses, some of which were wildly inaccurate. Much to his surprise, however, when he pooled the guesses of all 787 participants together, the crowd guessed the ox’s weight nearly perfectly.

From apes to ants, animals make better decisions when they work in groups than when they’re own their own. What is it about working in groups that makes us perform better? In a paper published this year in the Proceedings of the Royal Society B, Dr. Max Wolf and colleagues present one way that bigger groups can make better decisions.

Group of apes

We know that animals make better decisions in a group than by themselves, but we’re still trying to find out why.

Harnessing collective intelligence

In their paper, Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positives, the authors explain that groups can harness each members’ unique perspective on a problem by following a specific decision-making process. First, the group members must examine the problem themselves and secretly vote “yes” or “no.” (In the paper, “yes” or “no” indicates whether they saw a predator in a picture.) Next, the group members must pool their votes and vote for a second and final time, but this time they vote “yes” only if a sufficient fraction of group members also voted yes, called the quorum threshold. The result of this decision-making process is shown in the figure below.

Collective cognition increases accuracy

Bigger groups make better decisions: Adding group members increases true positives (accurate “yes” votes) while decreasing false positives (inaccurate “yes” votes).

In this model, each group member by themselves votes “yes” correctly 60% of the time (true positive) and votes “yes” incorrectly 30% of the time (false positive). This means that, by themselves, each group member will perform pretty poorly. However, when they follow the decision-making process described above, a group with 65 members can improve their accuracy to nearly 100%! How’s that for the wisdom of the crowds?

Since the authors didn’t include the code with their paper, I hacked together a quick version of the model and put it up on Github. Feel free to copy and reuse it as you like.

Now, you may be wondering: “How the heck does that work?” How do otherwise mediocre decision makers make near-perfect decisions when they work together? Unfortunately, the authors don’t go into detail on the why of this decision-making process, which no doubt means that will be the focus of future work.

What the paper does explain, however, is the importance of the quorum threshold value. If you play around with the quorum threshold value in the model I linked above, you’ll quickly find that this phenomenon only occurs when the quorum threshold value is in between the true positive and false positive rates. If you set the quorum threshold value outside of that range, then adding more group members can actually make the group less accurate.

Models are great, but what about the real world?

What the authors did next is what really made this study impressive. Knowing that their model predicts the optimal quorum threshold value to fall between a group’s true positive and false positive rate, they conducted an experiment with humans to see if this prediction holds up in the real world. Sure enough, shown in the figure below, each group’s quorum threshold (labeled “escape quorum”) fell in between the group’s true positive and false positive rate.

Quorum threshold measured in humans

Quorum threshold measured in humans. Figure from http://dx.doi.org/10.1098/rspb.2012.2777.

The implications of this finding are profound: Not only does having an intermediate quorum threshold increase the efficacy of a group, but it appears that humans are adapted to have an intermediate quorum threshold! It will be interesting to see how this quorum threshold varies across different decision-making tasks. I also wonder if this finding will hold if we measure it in other animal species, or if this finding only applies to humans.

So, what does this mean for me?

All this theory and experimental findings are neat, but what does this mean for businesses, engineers, and the public at large? As it turns out, we all have to make important “yes” or “no” decisions every day. I’ll highlight a couple examples from the paper, then add a couple examples of my own. Feel free to add your own examples in the comments.

Doctor

When a doctor examines a patient, they have to decide whether or not the patient has a disease or other ailment. Correctly diagnosing diseases (true positive) while avoiding identifying healthy patients as sick (false positive) is a vital part of a doctor’s job. As such, doctors can make better decisions by independently diagnosing a patient, then pooling their diagnoses together to make a final diagnosis.

Search committee

Hiring a new employee can be an expensive and time-consuming process, so it’s important to choose the best employee for the position. Search committees can best assess their applicants by assessing each applicant individually before discussing the applicants as a group.

Engineer

When analyzing the blueprints for a building, it is vital for engineers to accurately identify any potential structural faults in the building plans. Thus, design firms are better off having multiple engineers analyze the blueprints independently before holding a meeting to discuss any faults in the plans. Further, it is likely that numerous less-experienced engineers will identify more faults than a single experienced engineer.

reddit

I don’t know if the reddit designers intended to capitalize on the “wisdom of the crowds” with the reddit model, but there’s no denying that they owe their success to it. reddit delivers the most interesting content on the internet because thousands of people are voting on each link, which no doubt leads to an incredibly accurate decision of whether each link is interesting and relevant to the subreddit.

The future of collective intelligence

Decision making lies at the core of nearly every problem we face nowadays. A century after its discovery, there’s no doubt that we still have a lot to learn from animal collective intelligence. I hope that we see more research in this field so we can better understand how and why animals work together in groups.

References

Wolf, M., Kurvers, R., Ward, A., Krause, S., & Krause, J. (2013). Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positives Proceedings of the Royal Society B: Biological Sciences, 280 (1756), 20122777-20122777 DOI: 10.1098/rspb.2012.2777

Randy is a PhD candidate in Michigan State University's Computer Science program. As a member of Dr. Chris Adami's research lab, he studies biologically-inspired artificial intelligence and evolutionary processes.

Posted in research, review Tagged with: , ,
7 comments on “Bigger groups make better decisions
  1. temgc says:

    Interesting post, Randy. I think one of the key points is that for this “collective wisdom” to work, you have individuals making isolated judgments prior to having them entered into a pool. I think it would be interesting to compare this to collaborative decision-making among members of H. sapiens, where inputs may be modified by various social and political considerations (egotism, clout, dominance hierarchies, fear of retaliation, fear of being ostracized, fear of being considered stupid, etc…). Being a cynic, I’ll leave you with the following demotivational poster: http://www.despair.com/meetings.html

    • Randy Olson says:

      Thanks Eric! I agree completely, and they mention this in the paper: The value of making an independent assessment first is that you take advantage of everyone’s unique perspective and approach to the problem. In that sense, it would seem experience (or lack thereof), fear of being ostracized, etc. are actually useful for group decision making.

      Thanks for the poster. It really brightened my day. ;-)

  2. Jackie says:

    Thanks for hacking together that code.

  3. Hexigo says:

    Hi,
    Great research and post! We would like to quote / summarize you (and link / reference back to you of course) at some stage in the future.

    We ( http://www.Hexigo.com ) are a group decision making tool that empowers collective, collaborative decision making.

    We’ve touched on the benefits of group and collaborative decision making as you can see from some of our posts…..

    http://hexigo.com/group-decision-making-benefits

    http://hexigo.com/psychology-of-group-decisions

    But the quality of your research and articles is fantastic!

    Keep it up!

  4. The collective behaviour of animal groups is often decentralized, with no leader integrating different sources of information or telling the others what to do (Seeley 1995 , 2002 ). Instead, a pattern emerges from a large number of strictly local interactions that carry information throughout the group. A key feature of these interactions is positive feedback, in which an animal’s probability of exhibiting a particular behaviour is an increasing function of the number of conspecifics already performing this behaviour ( Deneubourg & Goss 1989 ; Bonabeau et al. 1997 ). In the context of collective decision-making, positive feedback allows the selection of a particular option to cascade through the group, as the growing number of adherents to an option increases its attractiveness to undecided animals. Moreover, this imitative behaviour often takes a step-like form, with an individual’s probability of selecting an option changing sharply when the number of like-minded conspecifics crosses a threshold. Here we refer to this functional form as a quorum response, following well-studied cases in which threshold group sizes trigger key changes in behaviour ( Pratt et al. 2002 ; Seeley & Visscher 2004b ).

1 Pings/Trackbacks for "Bigger groups make better decisions"
  1. […] each member’s unique perspective on a problem by following a specific decision-making process.”[1] Collectively, a group will do a better job of decision-making than an […]

About this blog

The data visualizations on this blog are the result of my “data tinkering” hobby, where I tackle a new data analysis problem every week. If I find something interesting, I report my findings here to share with the world.

If you like the work in this blog, I'm currently available for hire as a freelancer. Send me an email if you'd like to discuss freelance work.

If you would like to use one of my graphs on your website or in a publication, please email me. Donations to keep the site running ad-free are greatly appreciated, but never required.

Archives

Enter your email address to subscribe to this blog and receive notifications of new posts by email.