Opportunities and Risks of LLMs for Scalable Deliberation with Polis
Authors:
Christopher T. Small,
Ivan Vendrov,
Esin Durmus,
Hadjar Homaei,
Elizabeth Barry,
Julien Cornebise,
Ted Suzman,
Deep Ganguli,
Colin Megill
Abstract:
Polis is a platform that leverages machine intelligence to scale up deliberative processes. In this paper, we explore the opportunities and risks associated with applying Large Language Models (LLMs) towards challenges with facilitating, moderating and summarizing the results of Polis engagements. In particular, we demonstrate with pilot experiments using Anthropic's Claude that LLMs can indeed au…
▽ More
Polis is a platform that leverages machine intelligence to scale up deliberative processes. In this paper, we explore the opportunities and risks associated with applying Large Language Models (LLMs) towards challenges with facilitating, moderating and summarizing the results of Polis engagements. In particular, we demonstrate with pilot experiments using Anthropic's Claude that LLMs can indeed augment human intelligence to help more efficiently run Polis conversations. In particular, we find that summarization capabilities enable categorically new methods with immense promise to empower the public in collective meaning-making exercises. And notably, LLM context limitations have a significant impact on insight and quality of these results.
However, these opportunities come with risks. We discuss some of these risks, as well as principles and techniques for characterizing and mitigating them, and the implications for other deliberative or political systems that may employ LLMs. Finally, we conclude with several open future research directions for augmenting tools like Polis with LLMs.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
"Coherent Mode" for the World's Public Square
Authors:
Colin Megill,
Elizabeth Barry,
Christopher Small
Abstract:
Systems for large scale deliberation have resolved polarized issues and shifted agenda setting into the public's hands. These systems integrate bridging-based ranking algorithms - including group informed consensus implemented in Polis and the continuous matrix factorization approach implemented by Twitter Birdwatch - making it possible to highlight statements which enjoy broad support from a dive…
▽ More
Systems for large scale deliberation have resolved polarized issues and shifted agenda setting into the public's hands. These systems integrate bridging-based ranking algorithms - including group informed consensus implemented in Polis and the continuous matrix factorization approach implemented by Twitter Birdwatch - making it possible to highlight statements which enjoy broad support from a diversity of opinion groups.
Polis has been productively employed to foster more constructive political deliberation at nation scale in law making exercises. Twitter Birdwatch is implemented with the intention of addressing misinformation in the global public square. From one perspective, Twitter Birdwatch can be viewed as an anti-misinformation system which has deliberative aspects. But it can also be viewed as a first step towards a generalized deliberative system, using Twitter's misinformation problem as a proving ground.
In this paper, we propose that Twitter could adapt Birdwatch to produce maps of public opinion. We describe a system in five parts for generalizing Birdwatch: activation of a deliberative system and topic selection, population sampling and the role of expert networks, deliberation, reporting interpretable results and finally distribution of the results to the public and those in power.
△ Less
Submitted 22 November, 2022;
originally announced November 2022.
