-
Towards Coding Social Science Datasets with Language Models
Authors:
Christopher Michael Rytting,
Taylor Sorensen,
Lisa Argyle,
Ethan Busby,
Nancy Fulda,
Joshua Gubler,
David Wingate
Abstract:
Researchers often rely on humans to code (label, annotate, etc.) large sets of texts. This kind of human coding forms an important part of social science research, yet the coding process is both resource intensive and highly variable from application to application. In some cases, efforts to automate this process have achieved human-level accuracies, but to achieve this, these attempts frequently…
▽ More
Researchers often rely on humans to code (label, annotate, etc.) large sets of texts. This kind of human coding forms an important part of social science research, yet the coding process is both resource intensive and highly variable from application to application. In some cases, efforts to automate this process have achieved human-level accuracies, but to achieve this, these attempts frequently rely on thousands of hand-labeled training examples, which makes them inapplicable to small-scale research studies and costly for large ones. Recent advances in a specific kind of artificial intelligence tool - language models (LMs) - provide a solution to this problem. Work in computer science makes it clear that LMs are able to classify text, without the cost (in financial terms and human effort) of alternative methods. To demonstrate the possibilities of LMs in this area of political science, we use GPT-3, one of the most advanced LMs, as a synthetic coder and compare it to human coders. We find that GPT-3 can match the performance of typical human coders and offers benefits over other machine learning methods of coding text. We find this across a variety of domains using very different coding procedures. This provides exciting evidence that language models can serve as a critical advance in the coding of open-ended texts in a variety of applications.
△ Less
Submitted 3 June, 2023;
originally announced June 2023.
-
AI Chat Assistants can Improve Conversations about Divisive Topics
Authors:
Lisa P. Argyle,
Ethan Busby,
Joshua Gubler,
Chris Bail,
Thomas Howe,
Christopher Rytting,
David Wingate
Abstract:
A rapidly increasing amount of human conversation occurs online. But divisiveness and conflict can fester in text-based interactions on social media platforms, in messaging apps, and on other digital forums. Such toxicity increases polarization and, importantly, corrodes the capacity of diverse societies to develop efficient solutions to complex social problems that impact everyone. Scholars and c…
▽ More
A rapidly increasing amount of human conversation occurs online. But divisiveness and conflict can fester in text-based interactions on social media platforms, in messaging apps, and on other digital forums. Such toxicity increases polarization and, importantly, corrodes the capacity of diverse societies to develop efficient solutions to complex social problems that impact everyone. Scholars and civil society groups promote interventions that can make interpersonal conversations less divisive or more productive in offline settings, but scaling these efforts to the amount of discourse that occurs online is extremely challenging. We present results of a large-scale experiment that demonstrates how online conversations about divisive topics can be improved with artificial intelligence tools. Specifically, we employ a large language model to make real-time, evidence-based recommendations intended to improve participants' perception of feeling understood in conversations. We find that these interventions improve the reported quality of the conversation, reduce political divisiveness, and improve the tone, without systematically changing the content of the conversation or moving people's policy attitudes. These findings have important implications for future research on social media, political deliberation, and the growing community of scholars interested in the place of artificial intelligence within computational social science.
△ Less
Submitted 20 October, 2023; v1 submitted 14 February, 2023;
originally announced February 2023.
-
L 363-38 b: a planet newly discovered with ESPRESSO orbiting a nearby M dwarf star
Authors:
Lia F. Sartori,
Christophe Lovis,
Jean-Baptiste Delisle,
Monika Lendl,
Gabriele Cugno,
Anna Boehle,
Felix Dannert,
Andrea Krenn,
Jonas L. Gubler,
Sascha P. Quanz
Abstract:
Context. Planets around stars in the solar neighbourhood will be prime targets for characterisation with upcoming large space- and ground-based facilities. Since large-scale exoplanet searches will not be feasible with such telescopes, it is crucial to use currently available data and instruments to find possible target planets before next generation facilities come online.
Aims. We aim at detec…
▽ More
Context. Planets around stars in the solar neighbourhood will be prime targets for characterisation with upcoming large space- and ground-based facilities. Since large-scale exoplanet searches will not be feasible with such telescopes, it is crucial to use currently available data and instruments to find possible target planets before next generation facilities come online.
Aims. We aim at detecting new extrasolar planets around stars in the solar neighbourhood by blind radial velocity (RV) search with ESPRESSO. Our target sample consist of nearby stars (d < 11 pc) with little (< 10) or no previous RV measurements.
Methods. We use 31 radial velocity measurements obtained with ESPRESSO at the VLT between December 2020 and February 2022 of the nearby M dwarf star (M_star = 0.21 M_sun, d = 10.23 pc) L 363-38 to derive the orbital parameters of the newly discovered planet. In addition, we use TESS photometry and archival VLT/NaCo high contrast imaging data to put further constraints on the orbit inclination and the possible planetary system architecture around L 363-38.
Results. We present the detection of a new extrasolar planet orbiting the nearby M dwarf star L 363-38. L 363-38 b is a planet with minimum mass mp sin(i) = 4.67+/-0.43 M_Earth orbiting its star with a period P = 8.781+/-0.007 d, corresponding to a semi-major axis a = 0.048+/-0.006 AU, which is well inside the inner edge of the habitable zone. We further estimate a minimum radius rp sin(i) = 1.55 - 2.75 R_Earth and an equilibrium temperature Teq = 330K.
△ Less
Submitted 23 October, 2022;
originally announced October 2022.
-
Out of One, Many: Using Language Models to Simulate Human Samples
Authors:
Lisa P. Argyle,
Ethan C. Busby,
Nancy Fulda,
Joshua Gubler,
Christopher Rytting,
David Wingate
Abstract:
We propose and explore the possibility that language models can be studied as effective proxies for specific human sub-populations in social science research. Practical and research applications of artificial intelligence tools have sometimes been limited by problematic biases (such as racism or sexism), which are often treated as uniform properties of the models. We show that the "algorithmic bia…
▽ More
We propose and explore the possibility that language models can be studied as effective proxies for specific human sub-populations in social science research. Practical and research applications of artificial intelligence tools have sometimes been limited by problematic biases (such as racism or sexism), which are often treated as uniform properties of the models. We show that the "algorithmic bias" within one such tool -- the GPT-3 language model -- is instead both fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups. We term this property "algorithmic fidelity" and explore its extent in GPT-3. We create "silicon samples" by conditioning the model on thousands of socio-demographic backstories from real human participants in multiple large surveys conducted in the United States. We then compare the silicon and human samples to demonstrate that the information contained in GPT-3 goes far beyond surface similarity. It is nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and socio-cultural context that characterize human attitudes. We suggest that language models with sufficient algorithmic fidelity thus constitute a novel and powerful tool to advance understanding of humans and society across a variety of disciplines.
△ Less
Submitted 14 September, 2022;
originally announced September 2022.
