-
Deliberative Technology for Alignment
Authors:
Andrew Konya,
Deger Turan,
Aviv Ovadya,
Lina Qui,
Daanish Masood,
Flynn Devine,
Lisa Schirch,
Isabella Roberts,
Deliberative Alignment Forum
Abstract:
For humanity to maintain and expand its agency into the future, the most powerful systems we create must be those which act to align the future with the will of humanity. The most powerful systems today are massive institutions like governments, firms, and NGOs. Deliberative technology is already being used across these institutions to help align governance and diplomacy with human will, and moder…
▽ More
For humanity to maintain and expand its agency into the future, the most powerful systems we create must be those which act to align the future with the will of humanity. The most powerful systems today are massive institutions like governments, firms, and NGOs. Deliberative technology is already being used across these institutions to help align governance and diplomacy with human will, and modern AI is poised to make this technology significantly better. At the same time, the race to superhuman AGI is already underway, and the AI systems it gives rise to may become the most powerful systems of the future. Failure to align the impact of such powerful AI with the will of humanity may lead to catastrophic consequences, while success may unleash abundance. Right now, there is a window of opportunity to use deliberative technology to align the impact of powerful AI with the will of humanity. Moreover, it may be possible to engineer a symbiotic coupling between powerful AI and deliberative alignment systems such that the quality of alignment improves as AI capabilities increase.
△ Less
Submitted 6 December, 2023;
originally announced December 2023.
-
Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives
Authors:
Elizabeth Seger,
Noemi Dreksler,
Richard Moulange,
Emily Dardaman,
Jonas Schuett,
K. Wei,
Christoph Winter,
Mackenzie Arnold,
Seán Ó hÉigeartaigh,
Anton Korinek,
Markus Anderljung,
Ben Bucknall,
Alan Chan,
Eoghan Stafford,
Leonie Koessler,
Aviv Ovadya,
Ben Garfinkel,
Emma Bluemke,
Michael Aird,
Patrick Levermore,
Julian Hazell,
Abhishek Gupta
Abstract:
Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI models should be shared. Open-sourcing in AI typically refers to making model architecture and weights freely and publicly accessible for anyone to modify, study, build on, and use. This offers advantages such as enabling ex…
▽ More
Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI models should be shared. Open-sourcing in AI typically refers to making model architecture and weights freely and publicly accessible for anyone to modify, study, build on, and use. This offers advantages such as enabling external oversight, accelerating progress, and decentralizing control over AI development and use. However, it also presents a growing potential for misuse and unintended consequences. This paper offers an examination of the risks and benefits of open-sourcing highly capable foundation models. While open-sourcing has historically provided substantial net benefits for most software and AI development processes, we argue that for some highly capable foundation models likely to be developed in the near future, open-sourcing may pose sufficiently extreme risks to outweigh the benefits. In such a case, highly capable foundation models should not be open-sourced, at least not initially. Alternative strategies, including non-open-source model sharing options, are explored. The paper concludes with recommendations for developers, standard-setting bodies, and governments for establishing safe and responsible model sharing practices and preserving open-source benefits where safe.
△ Less
Submitted 29 September, 2023;
originally announced November 2023.
-
Democratic Policy Development using Collective Dialogues and AI
Authors:
Andrew Konya,
Lisa Schirch,
Colin Irwin,
Aviv Ovadya
Abstract:
We design and test an efficient democratic process for develo** policies that reflect informed public will. The process combines AI-enabled collective dialogues that make deliberation democratically viable at scale with bridging-based ranking for automated consensus discovery. A GPT4-powered pipeline translates points of consensus into representative policy clauses from which an initial policy i…
▽ More
We design and test an efficient democratic process for develo** policies that reflect informed public will. The process combines AI-enabled collective dialogues that make deliberation democratically viable at scale with bridging-based ranking for automated consensus discovery. A GPT4-powered pipeline translates points of consensus into representative policy clauses from which an initial policy is assembled. The initial policy is iteratively refined with the input of experts and the public before a final vote and evaluation. We test the process three times with the US public, develo** policy guidelines for AI assistants related to medical advice, vaccine information, and wars & conflicts. We show the process can be run in two weeks with 1500+ participants for around $10,000, and that it generates policy guidelines with strong public support across demographic divides. We measure 75-81% support for the policy guidelines overall, and no less than 70-75% support across demographic splits spanning age, gender, religion, race, education, and political party. Overall, this work demonstrates an end-to-end proof of concept for a process we believe can help AI labs develop common-ground policies, governing bodies break political gridlock, and diplomats accelerate peace deals.
△ Less
Submitted 3 November, 2023;
originally announced November 2023.
-
Democratising AI: Multiple Meanings, Goals, and Methods
Authors:
Elizabeth Seger,
Aviv Ovadya,
Ben Garfinkel,
Divya Siddarth,
Allan Dafoe
Abstract:
Numerous parties are calling for the democratisation of AI, but the phrase is used to refer to a variety of goals, the pursuit of which sometimes conflict. This paper identifies four kinds of AI democratisation that are commonly discussed: (1) the democratisation of AI use, (2) the democratisation of AI development, (3) the democratisation of AI profits, and (4) the democratisation of AI governanc…
▽ More
Numerous parties are calling for the democratisation of AI, but the phrase is used to refer to a variety of goals, the pursuit of which sometimes conflict. This paper identifies four kinds of AI democratisation that are commonly discussed: (1) the democratisation of AI use, (2) the democratisation of AI development, (3) the democratisation of AI profits, and (4) the democratisation of AI governance. Numerous goals and methods of achieving each form of democratisation are discussed. The main takeaway from this paper is that AI democratisation is a multifarious and sometimes conflicting concept that should not be conflated with improving AI accessibility. If we want to move beyond ambiguous commitments to democratising AI, to productive discussions of concrete policies and trade-offs, then we need to recognise the principal role of the democratisation of AI governance in navigating tradeoffs and risks across decisions around use, development, and profits.
△ Less
Submitted 7 August, 2023; v1 submitted 22 March, 2023;
originally announced March 2023.
-
'Generative CI' through Collective Response Systems
Authors:
Aviv Ovadya
Abstract:
How can many people (who may disagree) come together to answer a question or make a decision? "Collective response systems" are a type of generative collective intelligence (CI) facilitation process meant to address this challenge. They enable a form of "generative voting", where both the votes, and the choices of what to vote on, are provided by the group. Such systems overcome the traditional li…
▽ More
How can many people (who may disagree) come together to answer a question or make a decision? "Collective response systems" are a type of generative collective intelligence (CI) facilitation process meant to address this challenge. They enable a form of "generative voting", where both the votes, and the choices of what to vote on, are provided by the group. Such systems overcome the traditional limitations of polling, town halls, standard voting, referendums, etc. The generative CI outputs of collective response systems can also be chained together into iterative "collective dialogues", analogously to some kinds of generative AI.
Technical advances across domains including recommender systems, language models, and human-computer interaction have led to the development of innovative and scalable collective response systems. For example, Polis has been used around the world to support policy-making at different levels of government, and Remesh has been used by the UN to understand the challenges and needs of ordinary people across war-torn countries. This paper aims to develop a shared language by defining the structure, processes, properties, and principles of such systems.
Collective response systems allow non-confrontational exploration of divisive issues, help identify common ground, and elicit insights from those closest to the issues. As a result, they can help overcome gridlock around conflict and governance challenges, increase trust, and develop mandates. Continued progress toward their development and adoption could help revitalize democracies, reimagine corporate governance, transform conflict, and govern powerful AI systems -- both as a complement to deeper deliberative democratic processes and as an option where deeper processes are not applicable or possible.
△ Less
Submitted 1 February, 2023;
originally announced February 2023.
-
Bridging Systems: Open Problems for Countering Destructive Divisiveness across Ranking, Recommenders, and Governance
Authors:
Aviv Ovadya,
Luke Thorburn
Abstract:
Divisiveness appears to be increasing in much of the world, leading to concern about political violence and a decreasing capacity to collaboratively address large-scale societal challenges. In this working paper we aim to articulate an interdisciplinary research and practice area focused on what we call bridging systems: systems which increase mutual understanding and trust across divides, creatin…
▽ More
Divisiveness appears to be increasing in much of the world, leading to concern about political violence and a decreasing capacity to collaboratively address large-scale societal challenges. In this working paper we aim to articulate an interdisciplinary research and practice area focused on what we call bridging systems: systems which increase mutual understanding and trust across divides, creating space for productive conflict, deliberation, or cooperation. We give examples of bridging systems across three domains: recommender systems on social media, collective response systems, and human-facilitated group deliberation. We argue that these examples can be more meaningfully understood as processes for attention-allocation (as opposed to "content distribution" or "amplification") and develop a corresponding framework to explore similarities - and opportunities for bridging - across these seemingly disparate domains. We focus particularly on the potential of bridging-based ranking to bring the benefits of offline bridging into spaces which are already governed by algorithms. Throughout, we suggest research directions that could improve our capacity to incorporate bridging into a world increasingly mediated by algorithms and artificial intelligence.
△ Less
Submitted 24 July, 2023; v1 submitted 24 January, 2023;
originally announced January 2023.
-
The Deepfake Detection Dilemma: A Multistakeholder Exploration of Adversarial Dynamics in Synthetic Media
Authors:
Claire Leibowicz,
Sean McGregor,
Aviv Ovadya
Abstract:
Synthetic media detection technologies label media as either synthetic or non-synthetic and are increasingly used by journalists, web platforms, and the general public to identify misinformation and other forms of problematic content. As both well-resourced organizations and the non-technical general public generate more sophisticated synthetic media, the capacity for purveyors of problematic cont…
▽ More
Synthetic media detection technologies label media as either synthetic or non-synthetic and are increasingly used by journalists, web platforms, and the general public to identify misinformation and other forms of problematic content. As both well-resourced organizations and the non-technical general public generate more sophisticated synthetic media, the capacity for purveyors of problematic content to adapt induces a \newterm{detection dilemma}: as detection practices become more accessible, they become more easily circumvented. This paper describes how a multistakeholder cohort from academia, technology platforms, media entities, and civil society organizations active in synthetic media detection and its socio-technical implications evaluates the detection dilemma. Specifically, we offer an assessment of detection contexts and adversary capacities sourced from the broader, global AI and media integrity community concerned with mitigating the spread of harmful synthetic media. A collection of personas illustrates the intersection between unsophisticated and highly-resourced sponsors of misinformation in the context of their technical capacities. This work concludes that there is no "best" approach to navigating the detector dilemma, but derives a set of implications from multistakeholder input to better inform detection process decisions and policies, in practice.
△ Less
Submitted 11 February, 2021;
originally announced February 2021.
-
The tension between openness and prudence in AI research
Authors:
Jess Whittlestone,
Aviv Ovadya
Abstract:
This paper explores the tension between openness and prudence in AI research, evident in two core principles of the Montréal Declaration for Responsible AI. While the AI community has strong norms around open sharing of research, concerns about the potential harms arising from misuse of research are growing, prompting some to consider whether the field of AI needs to reconsider publication norms.…
▽ More
This paper explores the tension between openness and prudence in AI research, evident in two core principles of the Montréal Declaration for Responsible AI. While the AI community has strong norms around open sharing of research, concerns about the potential harms arising from misuse of research are growing, prompting some to consider whether the field of AI needs to reconsider publication norms. We discuss how different beliefs and values can lead to differing perspectives on how the AI community should manage this tension, and explore implications for what responsible publication norms in AI research might look like in practice.
△ Less
Submitted 13 January, 2020; v1 submitted 2 October, 2019;
originally announced October 2019.
-
Cross-Router Covert Channels
Authors:
Adar Ovadya,
Rom Ogen,
Yakov Mallah,
Niv Gilboa,
Yossi Oren
Abstract:
Many organizations protect secure networked devices from non-secure networked devices by assigning each class of devices to a different logical network. These two logical networks, commonly called the host network and the guest network, use the same router hardware, which is designed to isolate the two networks in software.
In this work we show that logical network isolation based on host and gu…
▽ More
Many organizations protect secure networked devices from non-secure networked devices by assigning each class of devices to a different logical network. These two logical networks, commonly called the host network and the guest network, use the same router hardware, which is designed to isolate the two networks in software.
In this work we show that logical network isolation based on host and guest networks can be overcome by the use of cross-router covert channels. Using specially-crafted network traffic, these channels make it possible to leak data between the host network and the guest network, and vice versa, through the use of the router as a shared medium. We performed a survey of routers representing multiple vendors and price points, and discovered that all of the routers we surveyed are vulnerable to at least one class of covert channel. Our attack can succeed even if the attacker has very limited permissions on the infected device, and even an iframe hosting malicious JavaScript code can be used for this purpose. We provide several metrics for the effectiveness of such channels, based on their pervasiveness, rate and covertness, and discuss possible ways of identifying and preventing these leakages.
△ Less
Submitted 7 August, 2019;
originally announced August 2019.
-
Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning
Authors:
Aviv Ovadya,
Jess Whittlestone
Abstract:
The aim of this paper is to facilitate nuanced discussion around research norms and practices to mitigate the harmful impacts of advances in machine learning (ML). We focus particularly on the use of ML to create "synthetic media" (e.g. to generate or manipulate audio, video, images, and text), and the question of what publication and release processes around such research might look like, though…
▽ More
The aim of this paper is to facilitate nuanced discussion around research norms and practices to mitigate the harmful impacts of advances in machine learning (ML). We focus particularly on the use of ML to create "synthetic media" (e.g. to generate or manipulate audio, video, images, and text), and the question of what publication and release processes around such research might look like, though many of the considerations discussed will apply to ML research more broadly. We are not arguing for any specific approach on when or how research should be distributed, but instead try to lay out some useful tools, analogies, and options for thinking about these issues.
We begin with some background on the idea that ML research might be misused in harmful ways, and why advances in synthetic media, in particular, are raising concerns. We then outline in more detail some of the different paths to harm from ML research, before reviewing research risk mitigation strategies in other fields and identifying components that seem most worth emulating in the ML and synthetic media research communities. Next, we outline some important dimensions of disagreement on these issues which risk polarizing conversations.
Finally, we conclude with recommendations, suggesting that the machine learning community might benefit from: working with subject matter experts to increase understanding of the risk landscape and possible mitigation strategies; building a community and norms around understanding the impacts of ML research, e.g. through regular workshops at major conferences; and establishing institutions and systems to support release practices that would otherwise be onerous and error-prone.
△ Less
Submitted 28 July, 2019; v1 submitted 25 July, 2019;
originally announced July 2019.
-
A Universal Crease Pattern for Folding Orthogonal Shapes
Authors:
Nadia Benbernou,
Erik D. Demaine,
Martin L. Demaine,
Aviv Ovadya
Abstract:
We present a universal crease pattern--known in geometry as the tetrakis tiling and in origami as box pleating--that can fold into any object made up of unit cubes joined face-to-face (polycubes). More precisely, there is one universal finite crease pattern for each number n of unit cubes that need to be folded. This result contrasts previous universality results for origami, which require a dif…
▽ More
We present a universal crease pattern--known in geometry as the tetrakis tiling and in origami as box pleating--that can fold into any object made up of unit cubes joined face-to-face (polycubes). More precisely, there is one universal finite crease pattern for each number n of unit cubes that need to be folded. This result contrasts previous universality results for origami, which require a different crease pattern for each target object, and confirms intuition in the origami community that box pleating is a powerful design technique.
△ Less
Submitted 29 September, 2009;
originally announced September 2009.
