-
Communication Reliably Improves Individual But Not Group Accuracy
Authors:
Charlie Pilgrim,
Joshua Becker
Abstract:
Prior research offers mixed evidence on whether and when communication improves belief accuracy for numeric estimates. Experiments on one-to-one advice suggest that communication between peers usually benefits accuracy, while group experiments indicate that communication networks produce highly variable outcomes. Notably, it is possible for a group's average estimate to become less accurate even a…
▽ More
Prior research offers mixed evidence on whether and when communication improves belief accuracy for numeric estimates. Experiments on one-to-one advice suggest that communication between peers usually benefits accuracy, while group experiments indicate that communication networks produce highly variable outcomes. Notably, it is possible for a group's average estimate to become less accurate even as its individual group members -- on average -- become more accurate. However, the conditions under which communication improves group and/or individual outcomes remain poorly characterised. We analyse an empirically supported model of opinion formation to derive these conditions, formally explicating the relationship between group-level effects and individual outcomes. We reanalyze previously published experimental data, finding that empirical dynamics are consistent with theoretical expectations. We show that 3 measures completely describe asymptotic opinion dynamics: the initial crowd bias; the degree of influence centralisation; and the correlation between influence and initial biases. We find analytic expressions for the change in crowd and individual accuracy as a function of the product of these three measures, which we describe as the truth alignment. We show how truth alignment can be decomposed into calibration (influence/accuracy correlation), and herding (influence/averageness correlation), and how these measures relate to changes in accuracy. Overall, we find that individuals can and usually do improve even when groups get worse.
△ Less
Submitted 28 June, 2024;
originally announced July 2024.
-
The Crowd Classification Problem: Social Dynamics of Binary Choice Accuracy
Authors:
Joshua Becker,
Douglas Guilbeault,
Ned Smith
Abstract:
Decades of research suggest that information exchange in groups and organizations can reliably improve judgment accuracy in tasks such as financial forecasting, market research, and medical decision-making. However, we show that improving the accuracy of numeric estimates does not necessarily improve the accuracy of decisions. For binary choice judgments, also known as classification tasks--e.g. y…
▽ More
Decades of research suggest that information exchange in groups and organizations can reliably improve judgment accuracy in tasks such as financial forecasting, market research, and medical decision-making. However, we show that improving the accuracy of numeric estimates does not necessarily improve the accuracy of decisions. For binary choice judgments, also known as classification tasks--e.g. yes/no or build/buy decisions--social influence is most likely to grow the majority vote share, regardless of the accuracy of that opinion. As a result, initially inaccurate groups become increasingly inaccurate after information exchange even as they signal stronger support. We term this dynamic the "crowd classification problem." Using both a novel dataset as well as a reanalysis of three previous datasets, we study this process in two types of information exchange: (1) when people share votes only, and (2) when people form and exchange numeric estimates prior to voting. Surprisingly, when people exchange numeric estimates prior to voting, the binary choice vote can become less accurate even as the average numeric estimate becomes more accurate. Our findings recommend against voting as a form of decision-making when groups are optimizing for accuracy. For those cases where voting is required, we discuss strategies for managing communication to avoid the crowd classification problem. We close with a discussion of how our results contribute to a broader contingency theory of collective intelligence.
△ Less
Submitted 22 April, 2021;
originally announced April 2021.
-
Network Structures of Collective Intelligence: The Contingent Benefits of Group Discussion
Authors:
Joshua Becker,
Abdullah Almaatouq,
Emőke-Ágnes Horvát
Abstract:
Research on belief formation has produced contradictory findings on whether and when communication between group members will improve the accuracy of numeric estimates such as economic forecasts, medical diagnoses, and job candidate assessments. While some evidence suggests that carefully mediated processes such as the "Delphi method" produce more accurate beliefs than unstructured discussion, oth…
▽ More
Research on belief formation has produced contradictory findings on whether and when communication between group members will improve the accuracy of numeric estimates such as economic forecasts, medical diagnoses, and job candidate assessments. While some evidence suggests that carefully mediated processes such as the "Delphi method" produce more accurate beliefs than unstructured discussion, others argue that unstructured discussion outperforms mediated processes. Still others argue that independent individuals produce the most accurate beliefs. This paper shows how network theories of belief formation can resolve these inconsistencies, even when groups lack apparent structure as in informal conversation. Emergent network structures of influence interact with the pre-discussion belief distribution to moderate the effect of communication on belief formation. As a result, communication sometimes increases and sometimes decreases the accuracy of the average belief in a group. The effects differ for mediated processes and unstructured communication, such that the relative benefit of each communication format depends on both group dynamics as well as the statistical properties of pre-interaction beliefs. These results resolve contradictions in previous research and offer practical recommendations for teams and organizations.
△ Less
Submitted 8 March, 2021; v1 submitted 15 September, 2020;
originally announced September 2020.
-
Network Structure and Collective Intelligence in the Diffusion of Innovation
Authors:
Joshua Becker
Abstract:
When multiple innovations compete for adoption, historical chance leading to early advantage can generate lock-in effects that allow suboptimal innovations to succeed at the expense of superior alternatives. Research on the diffusion of innovafacetion has identified many possible sources of early advantage, but these mechanisms can benefit both optimal and suboptimal innovations. This paper moves…
▽ More
When multiple innovations compete for adoption, historical chance leading to early advantage can generate lock-in effects that allow suboptimal innovations to succeed at the expense of superior alternatives. Research on the diffusion of innovafacetion has identified many possible sources of early advantage, but these mechanisms can benefit both optimal and suboptimal innovations. This paper moves beyond chance-as-explanation to identify structural principles that systematically impact the likelihood that the optimal strategy will spread. A formal model of innovation diffusion shows that the network structure of organizational relationships can systematically impact the likelihood that widely adopted innovations will be payoff optimal. Building on prior diffusion research, this paper focuses on the role of central actors i.e. well-connected people or firms. While contagion models of diffusion highlight the benefits of central actors for spreading innovations further and faster, the present analysis reveals a dark side to this influence: the mere presence of central actors in a network increases rates of adoption but also increases the likelihood of suboptimal outcomes. This effect, however, does not represent a speed-optimality tradeoff, as dense networks are both fast and optimal. This finding is consistent with related research showing that network centralization undermines collective intelligence.
△ Less
Submitted 30 January, 2023; v1 submitted 26 March, 2020;
originally announced March 2020.
