-
Bayesian Active Learning in the Presence of Nuisance Parameters
Authors:
Sabina J. Sloman,
Ayush Bharti,
Julien Martinelli,
Samuel Kaski
Abstract:
In many settings, such as scientific inference, optimization, and transfer learning, the learner has a well-defined objective, which can be treated as estimation of a target parameter, and no intrinsic interest in characterizing the entire data-generating process. Usually, the learner must also contend with additional sources of uncertainty or variables -- with nuisance parameters. Bayesian active…
▽ More
In many settings, such as scientific inference, optimization, and transfer learning, the learner has a well-defined objective, which can be treated as estimation of a target parameter, and no intrinsic interest in characterizing the entire data-generating process. Usually, the learner must also contend with additional sources of uncertainty or variables -- with nuisance parameters. Bayesian active learning, or sequential optimal experimental design, can straightforwardly accommodate the presence of nuisance parameters, and so is a natural active learning framework for such problems. However, the introduction of nuisance parameters can lead to bias in the Bayesian learner's estimate of the target parameters, a phenomenon we refer to as negative interference. We characterize the threat of negative interference and how it fundamentally changes the nature of the Bayesian active learner's task. We show that the extent of negative interference can be extremely large, and that accurate estimation of the nuisance parameters is critical to reducing it. The Bayesian active learner is confronted with a dilemma: whether to spend a finite acquisition budget in pursuit of estimation of the target or of the nuisance parameters. Our setting encompasses Bayesian transfer learning as a special case, and our results shed light on the phenomenon of negative transfer between learning environments.
△ Less
Submitted 10 June, 2024; v1 submitted 23 October, 2023;
originally announced October 2023.
-
The science of fake news
Authors:
David M. J. Lazer,
Matthew A. Baum,
Yochai Benkler,
Adam J. Berinsky,
Kelly M. Greenhill,
Filippo Menczer,
Miriam J. Metzger,
Brendan Nyhan,
Gordon Pennycook,
David Rothschild,
Michael Schudson,
Steven A. Sloman,
Cass R. Sunstein,
Emily A. Thorson,
Duncan J. Watts,
Jonathan L. Zittrain
Abstract:
Fake news emerged as an apparent global problem during the 2016 U.S. Presidential election. Addressing it requires a multidisciplinary effort to define the nature and extent of the problem, detect fake news in real time, and mitigate its potentially harmful effects. This will require a better understanding of how the Internet spreads content, how people process news, and how the two interact. We r…
▽ More
Fake news emerged as an apparent global problem during the 2016 U.S. Presidential election. Addressing it requires a multidisciplinary effort to define the nature and extent of the problem, detect fake news in real time, and mitigate its potentially harmful effects. This will require a better understanding of how the Internet spreads content, how people process news, and how the two interact. We review the state of knowledge in these areas and discuss two broad potential mitigation strategies: better enabling individuals to identify fake news, and intervention within the platforms to reduce the attention given to fake news. The cooperation of Internet platforms (especially Facebook, Google, and Twitter) with researchers will be critical to understanding the scale of the issue and the effectiveness of possible interventions.
△ Less
Submitted 15 July, 2023;
originally announced July 2023.
-
Learning Relevant Contextual Variables Within Bayesian Optimization
Authors:
Julien Martinelli,
Ayush Bharti,
Armi Tiihonen,
S. T. John,
Louis Filstroff,
Sabina J. Sloman,
Patrick Rinke,
Samuel Kaski
Abstract:
Contextual Bayesian Optimization (CBO) efficiently optimizes black-box functions with respect to design variables, while simultaneously integrating contextual information regarding the environment, such as experimental conditions. However, the relevance of contextual variables is not necessarily known beforehand. Moreover, contextual variables can sometimes be optimized themselves at an additional…
▽ More
Contextual Bayesian Optimization (CBO) efficiently optimizes black-box functions with respect to design variables, while simultaneously integrating contextual information regarding the environment, such as experimental conditions. However, the relevance of contextual variables is not necessarily known beforehand. Moreover, contextual variables can sometimes be optimized themselves at an additional cost, a setting overlooked by current CBO algorithms. Cost-sensitive CBO would simply include optimizable contextual variables as part of the design variables based on their cost. Instead, we adaptively select a subset of contextual variables to include in the optimization, based on the trade-off between their relevance and the additional cost incurred by optimizing them compared to leaving them to be determined by the environment. We learn the relevance of contextual variables by sensitivity analysis of the posterior surrogate model while minimizing the cost of optimization by leveraging recent developments on early stop** for BO. We empirically evaluate our proposed Sensitivity-Analysis-Driven Contextual BO (SADCBO) method against alternatives on both synthetic and real-world experiments, together with extensive ablation studies, and demonstrate a consistent improvement across examples.
△ Less
Submitted 24 May, 2024; v1 submitted 23 May, 2023;
originally announced May 2023.
-
Gathering Strength, Gathering Storms: The One Hundred Year Study on Artificial Intelligence (AI100) 2021 Study Panel Report
Authors:
Michael L. Littman,
Ifeoma Ajunwa,
Guy Berger,
Craig Boutilier,
Morgan Currie,
Finale Doshi-Velez,
Gillian Hadfield,
Michael C. Horowitz,
Charles Isbell,
Hiroaki Kitano,
Karen Levy,
Terah Lyons,
Melanie Mitchell,
Julie Shah,
Steven Sloman,
Shannon Vallor,
Toby Walsh
Abstract:
In September 2021, the "One Hundred Year Study on Artificial Intelligence" project (AI100) issued the second report of its planned long-term periodic assessment of artificial intelligence (AI) and its impact on society. It was written by a panel of 17 study authors, each of whom is deeply rooted in AI research, chaired by Michael Littman of Brown University. The report, entitled "Gathering Strengt…
▽ More
In September 2021, the "One Hundred Year Study on Artificial Intelligence" project (AI100) issued the second report of its planned long-term periodic assessment of artificial intelligence (AI) and its impact on society. It was written by a panel of 17 study authors, each of whom is deeply rooted in AI research, chaired by Michael Littman of Brown University. The report, entitled "Gathering Strength, Gathering Storms," answers a set of 14 questions probing critical areas of AI development addressing the major risks and dangers of AI, its effects on society, its public perception and the future of the field. The report concludes that AI has made a major leap from the lab to people's lives in recent years, which increases the urgency to understand its potential negative effects. The questions were developed by the AI100 Standing Committee, chaired by Peter Stone of the University of Texas at Austin, consisting of a group of AI leaders with expertise in computer science, sociology, ethics, economics, and other disciplines.
△ Less
Submitted 27 October, 2022;
originally announced October 2022.
-
A Novel Corpus of Discourse Structure in Humans and Computers
Authors:
Babak Hemmatian,
Sheridan Feucht,
Rachel Avram,
Alexander Wey,
Muskaan Garg,
Kate Spitalnic,
Carsten Eickhoff,
Ellie Pavlick,
Bjorn Sandstede,
Steven Sloman
Abstract:
We present a novel corpus of 445 human- and computer-generated documents, comprising about 27,000 clauses, annotated for semantic clause types and coherence relations that allow for nuanced comparison of artificial and natural discourse modes. The corpus covers both formal and informal discourse, and contains documents generated using fine-tuned GPT-2 (Zellers et al., 2019) and GPT-3(Brown et al.,…
▽ More
We present a novel corpus of 445 human- and computer-generated documents, comprising about 27,000 clauses, annotated for semantic clause types and coherence relations that allow for nuanced comparison of artificial and natural discourse modes. The corpus covers both formal and informal discourse, and contains documents generated using fine-tuned GPT-2 (Zellers et al., 2019) and GPT-3(Brown et al., 2020). We showcase the usefulness of this corpus for detailed discourse analysis of text generation by providing preliminary evidence that less numerous, shorter and more often incoherent clause relations are associated with lower perceived quality of computer-generated narratives and arguments.
△ Less
Submitted 10 November, 2021;
originally announced November 2021.
-
Using Human-Guided Causal Knowledge for More Generalized Robot Task Planning
Authors:
Semir Tatlidil,
Yanqi Liu,
Emily Sheetz,
R. Iris Bahar,
Steven Sloman
Abstract:
A major challenge in research involving artificial intelligence (AI) is the development of algorithms that can find solutions to problems that can generalize to different environments and tasks. Unlike AI, humans are adept at finding solutions that can transfer. We hypothesize this is because their solutions are informed by causal models. We propose to use human-guided causal knowledge to help rob…
▽ More
A major challenge in research involving artificial intelligence (AI) is the development of algorithms that can find solutions to problems that can generalize to different environments and tasks. Unlike AI, humans are adept at finding solutions that can transfer. We hypothesize this is because their solutions are informed by causal models. We propose to use human-guided causal knowledge to help robots find solutions that can generalize to a new environment. We develop and test the feasibility of a language interface that naïve participants can use to communicate these causal models to a planner. We find preliminary evidence that participants are able to use our interface and generate causal models that achieve near-generalization. We outline an experiment aimed at testing far-generalization using our interface and describe our longer terms goals for these causal models.
△ Less
Submitted 9 October, 2021;
originally announced October 2021.
