-
Human-AI collectives produce the most accurate differential diagnoses
Authors:
N. Zöller,
J. Berger,
I. Lin,
N. Fu,
J. Komarneni,
G. Barabucci,
K. Laskowski,
V. Shia,
B. Harack,
E. A. Chu,
V. Trianni,
R. H. J. M. Kurvers,
S. M. Herzog
Abstract:
Artificial intelligence systems, particularly large language models (LLMs), are increasingly being employed in high-stakes decisions that impact both individuals and society at large, often without adequate safeguards to ensure safety, quality, and equity. Yet LLMs hallucinate, lack common sense, and are biased - shortcomings that may reflect LLMs' inherent limitations and thus may not be remedied…
▽ More
Artificial intelligence systems, particularly large language models (LLMs), are increasingly being employed in high-stakes decisions that impact both individuals and society at large, often without adequate safeguards to ensure safety, quality, and equity. Yet LLMs hallucinate, lack common sense, and are biased - shortcomings that may reflect LLMs' inherent limitations and thus may not be remedied by more sophisticated architectures, more data, or more human feedback. Relying solely on LLMs for complex, high-stakes decisions is therefore problematic. Here we present a hybrid collective intelligence system that mitigates these risks by leveraging the complementary strengths of human experience and the vast information processed by LLMs. We apply our method to open-ended medical diagnostics, combining 40,762 differential diagnoses made by physicians with the diagnoses of five state-of-the art LLMs across 2,133 medical cases. We show that hybrid collectives of physicians and LLMs outperform both single physicians and physician collectives, as well as single LLMs and LLM ensembles. This result holds across a range of medical specialties and professional experience, and can be attributed to humans' and LLMs' complementary contributions that lead to different kinds of errors. Our approach highlights the potential for collective human and machine intelligence to improve accuracy in complex, open-ended domains like medical diagnostics.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
Crowd control: Reducing individual estimation bias by sharing biased social information
Authors:
Bertrand Jayles,
Clément Sire,
Ralf H. J. M Kurvers
Abstract:
Cognitive biases are widespread in humans and animals alike, and can sometimes be reinforced by social interactions. One prime bias in judgment and decision-making is the human tendency to underestimate large quantities. Previous research on social influence in estimation tasks has generally focused on the impact of single estimates on individual and collective accuracy, showing that randomly shar…
▽ More
Cognitive biases are widespread in humans and animals alike, and can sometimes be reinforced by social interactions. One prime bias in judgment and decision-making is the human tendency to underestimate large quantities. Previous research on social influence in estimation tasks has generally focused on the impact of single estimates on individual and collective accuracy, showing that randomly sharing estimates does not reduce the underestimation bias. Here, we test a method of social information sharing that exploits the known relationship between the true value and the level of underestimation, and study if it can counteract the underestimation bias. We performed estimation experiments in which participants had to estimate a series of quantities twice, before and after receiving estimates from one or several group members. Our purpose was threefold: to study (i) whether restructuring the sharing of social information can reduce the underestimation bias, (ii) how the number of estimates received affects the sensitivity to social influence and estimation accuracy, and (iii) the mechanisms underlying the integration of multiple estimates. Our restructuring of social interactions successfully countered the underestimation bias. Moreover, we find that sharing more than one estimate also reduces the underestimation bias. Underlying our results are a human tendency to herd, to trust larger estimates than one's own more than smaller estimates, and to follow disparate social information less. Using a computational modelling approach, we demonstrate that these effects are indeed key to explain the experimental results. Overall, our results show that existing knowledge on biases can be used to dampen their negative effects and boost judgment accuracy, paving the way for combating other cognitive biases threatening collective systems.
△ Less
Submitted 11 January, 2022; v1 submitted 15 March, 2020;
originally announced March 2020.
-
Impact of sharing full versus averaged social information on social influence and estimation accuracy
Authors:
Bertrand Jayles,
Clément Sire,
Ralf H. J. M Kurvers
Abstract:
The recent developments of social networks and recommender systems have dramatically increased the amount of social information shared in human communities, challenging the human ability to process it. As a result, sharing aggregated forms of social information is becoming increasingly popular. However, it is unknown whether sharing aggregated information improves people's judgments more than shar…
▽ More
The recent developments of social networks and recommender systems have dramatically increased the amount of social information shared in human communities, challenging the human ability to process it. As a result, sharing aggregated forms of social information is becoming increasingly popular. However, it is unknown whether sharing aggregated information improves people's judgments more than sharing the full available information. Here, we compare the performance of groups in estimation tasks when social information is fully shared versus when it is first averaged and then shared. We find that improvements in estimation accuracy are comparable in both cases. However, our results reveal important differences in subjects' behaviour: (i) subjects follow the social information more when receiving an average than when receiving all estimates, and this effect increases with the number of estimates underlying the average; (ii) subjects follow the social information more when it is higher than their personal estimate than when it is lower. This effect is stronger when receiving all estimates than when receiving an average. We introduce a model that sheds light on these effects, and confirms their importance for explaining improvements in estimation accuracy in all treatments.
△ Less
Submitted 11 January, 2022; v1 submitted 15 March, 2020;
originally announced March 2020.
-
Proto-Cooperation: group hunting sailfish improve hunting success by alternating attacks on grou** prey
Authors:
James E Herbert-Read,
Pawel Romanczuk,
Stefan Krause,
Daniel Strömbom,
Pierre Couillaud,
Paolo Domenici,
Ralf H. J. M. Kurvers,
Stefano Marras,
John F Steffensen,
Alexander D. M. Wilson,
Jens Krause
Abstract:
We present evidence of a novel form of group hunting. Individual sailfish (Istiophorus platypterus) alternate attacks with other group members on their schooling prey (Sardinella aurita). While only 24% of attacks result in prey capture, multiple prey are injured in 95% of attacks, resulting in an increase of injured fish in the school with the number of attacks. How quickly prey are captured is p…
▽ More
We present evidence of a novel form of group hunting. Individual sailfish (Istiophorus platypterus) alternate attacks with other group members on their schooling prey (Sardinella aurita). While only 24% of attacks result in prey capture, multiple prey are injured in 95% of attacks, resulting in an increase of injured fish in the school with the number of attacks. How quickly prey are captured is positively correlated with the level of injury of the school, suggesting that hunters can benefit from other conspecifics' attacks on the prey. To explore this, we built a mathematical model capturing the dynamics of the hunt. We show that group hunting provides major efficiency gains (prey caught per unit time) for individuals in groups of up to 70 members. We also demonstrate that a free riding strategy, where some individuals wait until the prey are sufficiently injured before attacking, is only beneficial if the cost of attacking is high, and only then when waiting times are short. Our findings provide evidence that cooperative benefits can be realised through the facilitative effects of individuals' hunting actions without spatial coordination of attacks. Such "proto-cooperation" may be the pre-cursor to more complex group-hunting strategies.
△ Less
Submitted 31 December, 2016; v1 submitted 2 March, 2016;
originally announced March 2016.
