-
Formal Methods for the Informal Engineer: Workshop Recommendations
Authors:
Gopal Sarma,
James Koppel,
Gregory Malecha,
Patrick Schultz,
Eric Drexler,
Ramana Kumar,
Cody Roux,
Philip Zucker
Abstract:
Formal Methods for the Informal Engineer (FMIE) was a workshop held at the Broad Institute of MIT and Harvard in 2021 to explore the potential role of verified software in the biomedical software ecosystem. The motivation for organizing FMIE was the recognition that the life sciences and medicine are undergoing a transition from being passive consumers of software and AI/ML technologies to fundame…
▽ More
Formal Methods for the Informal Engineer (FMIE) was a workshop held at the Broad Institute of MIT and Harvard in 2021 to explore the potential role of verified software in the biomedical software ecosystem. The motivation for organizing FMIE was the recognition that the life sciences and medicine are undergoing a transition from being passive consumers of software and AI/ML technologies to fundamental drivers of new platforms, including those which will need to be mission and safety-critical. Drawing on conversations leading up to and during the workshop, we make five concrete recommendations to help software leaders organically incorporate tools, techniques, and perspectives from formal methods into their project planning and development trajectories.
△ Less
Submitted 1 April, 2021;
originally announced April 2021.
-
Integrative Biological Simulation, Neuropsychology, and AI Safety
Authors:
Gopal P. Sarma,
Adam Safron,
Nick J. Hay
Abstract:
We describe a biologically-inspired research agenda with parallel tracks aimed at AI and AI safety. The bottom-up component consists of building a sequence of biophysically realistic simulations of simple organisms such as the nematode $Caenorhabditis$ $elegans$, the fruit fly $Drosophila$ $melanogaster$, and the zebrafish $Danio$ $rerio$ to serve as platforms for research into AI algorithms and s…
▽ More
We describe a biologically-inspired research agenda with parallel tracks aimed at AI and AI safety. The bottom-up component consists of building a sequence of biophysically realistic simulations of simple organisms such as the nematode $Caenorhabditis$ $elegans$, the fruit fly $Drosophila$ $melanogaster$, and the zebrafish $Danio$ $rerio$ to serve as platforms for research into AI algorithms and system architectures. The top-down component consists of an approach to value alignment that grounds AI goal structures in neuropsychology, broadly considered. Our belief is that parallel pursuit of these tracks will inform the development of value-aligned AI systems that have been inspired by embodied organisms with sensorimotor integration. An important set of side benefits is that the research trajectories we describe here are grounded in long-standing intellectual traditions within existing research communities and funding structures. In addition, these research programs overlap with significant contemporary themes in the biological and psychological sciences such as data/model integration and reproducibility.
△ Less
Submitted 21 January, 2019; v1 submitted 6 November, 2018;
originally announced November 2018.
-
AI Safety and Reproducibility: Establishing Robust Foundations for the Neuropsychology of Human Values
Authors:
Gopal P. Sarma,
Nick J. Hay,
Adam Safron
Abstract:
We propose the creation of a systematic effort to identify and replicate key findings in neuropsychology and allied fields related to understanding human values. Our aim is to ensure that research underpinning the value alignment problem of artificial intelligence has been sufficiently validated to play a role in the design of AI systems.
We propose the creation of a systematic effort to identify and replicate key findings in neuropsychology and allied fields related to understanding human values. Our aim is to ensure that research underpinning the value alignment problem of artificial intelligence has been sufficiently validated to play a role in the design of AI systems.
△ Less
Submitted 8 September, 2018; v1 submitted 8 December, 2017;
originally announced December 2017.
-
Robust Computer Algebra, Theorem Proving, and Oracle AI
Authors:
Gopal P. Sarma,
Nick J. Hay
Abstract:
In the context of superintelligent AI systems, the term "oracle" has two meanings. One refers to modular systems queried for domain-specific tasks. Another usage, referring to a class of systems which may be useful for addressing the value alignment and AI control problems, is a superintelligent AI system that only answers questions. The aim of this manuscript is to survey contemporary research pr…
▽ More
In the context of superintelligent AI systems, the term "oracle" has two meanings. One refers to modular systems queried for domain-specific tasks. Another usage, referring to a class of systems which may be useful for addressing the value alignment and AI control problems, is a superintelligent AI system that only answers questions. The aim of this manuscript is to survey contemporary research problems related to oracles which align with long-term research goals of AI safety. We examine existing question answering systems and argue that their high degree of architectural heterogeneity makes them poor candidates for rigorous analysis as oracles. On the other hand, we identify computer algebra systems (CASs) as being primitive examples of domain-specific oracles for mathematics and argue that efforts to integrate computer algebra systems with theorem provers, systems which have largely been developed independent of one another, provide a concrete set of problems related to the notion of provable safety that has emerged in the AI safety community. We review approaches to interfacing CASs with theorem provers, describe well-defined architectural deficiencies that have been identified with CASs, and suggest possible lines of research and practical software projects for scientists interested in AI safety.
△ Less
Submitted 31 December, 2017; v1 submitted 8 August, 2017;
originally announced August 2017.
-
Brief Notes on Hard Takeoff, Value Alignment, and Coherent Extrapolated Volition
Authors:
Gopal P. Sarma
Abstract:
I make some basic observations about hard takeoff, value alignment, and coherent extrapolated volition, concepts which have been central in analyses of superintelligent AI systems.
I make some basic observations about hard takeoff, value alignment, and coherent extrapolated volition, concepts which have been central in analyses of superintelligent AI systems.
△ Less
Submitted 21 April, 2018; v1 submitted 3 April, 2017;
originally announced April 2017.
-
Doing Things Twice (Or Differently): Strategies to Identify Studies for Targeted Validation
Authors:
Gopal P. Sarma
Abstract:
The "reproducibility crisis" has been a highly visible source of scientific controversy and dispute. Here, I propose and review several avenues for identifying and prioritizing research studies for the purpose of targeted validation. Of the various proposals discussed, I identify scientific data science as being a strategy that merits greater attention among those interested in reproducibility. I…
▽ More
The "reproducibility crisis" has been a highly visible source of scientific controversy and dispute. Here, I propose and review several avenues for identifying and prioritizing research studies for the purpose of targeted validation. Of the various proposals discussed, I identify scientific data science as being a strategy that merits greater attention among those interested in reproducibility. I argue that the tremendous potential of scientific data science for uncovering high-value research studies is a significant and rarely discussed benefit of the transition to a fully open-access publishing model.
△ Less
Submitted 21 April, 2018; v1 submitted 5 March, 2017;
originally announced March 2017.
-
Scientific Literature Text Mining and the Case for Open Access
Authors:
Gopal P. Sarma
Abstract:
"Open access" has become a central theme of journal reform in academic publishing. In this article, I examine the relationship between open access publishing and an important infrastructural element of a modern research enterprise, scientific literature text mining, or the use of data analytic techniques to conduct meta-analyses and investigations into the scientific corpus. I give a brief history…
▽ More
"Open access" has become a central theme of journal reform in academic publishing. In this article, I examine the relationship between open access publishing and an important infrastructural element of a modern research enterprise, scientific literature text mining, or the use of data analytic techniques to conduct meta-analyses and investigations into the scientific corpus. I give a brief history of the open access movement, discuss novel journalistic practices, and an overview of data-driven investigation of the scientific corpus. I argue that particularly in an era where the veracity of many research studies has been called into question, scientific literature text mining should be one of the key motivations for open access publishing, not only in the basic sciences, but in the engineering and applied sciences as well. The enormous benefits of unrestricted access to the research literature should prompt scholars from all disciplines to lend their vocal support to enabling legal, wholesale access to the scientific literature as part of a data science pipeline.
△ Less
Submitted 23 April, 2018; v1 submitted 31 October, 2016;
originally announced November 2016.
-
Scientific Auditing Firms
Authors:
Gopal P. Sarma
Abstract:
The "crisis of reproducibility" has been a significant source of controversy, heated debate, and calls for reform to institutional science in recent years. As a long-term solution to address both the present crisis and future obstacles, I propose the creation of a new form of research organization whose purpose would be to conduct random audits of the scientific literature. I suggest that data ana…
▽ More
The "crisis of reproducibility" has been a significant source of controversy, heated debate, and calls for reform to institutional science in recent years. As a long-term solution to address both the present crisis and future obstacles, I propose the creation of a new form of research organization whose purpose would be to conduct random audits of the scientific literature. I suggest that data analytics of a digitized scientific corpus may play a critical role in allowing broadly educated scientists to identify linchpin results to investigate in further detail across all disciplines. I argue that a simple "mock" trial run of a simplified auditing firm consisting of several researchers over a short time period would provide valuable insight into the feasibility of this proposal.
△ Less
Submitted 23 April, 2018; v1 submitted 1 August, 2016;
originally announced August 2016.
-
Mammalian Value Systems
Authors:
Gopal P. Sarma,
Nick J. Hay
Abstract:
Characterizing human values is a topic deeply interwoven with the sciences, humanities, art, and many other human endeavors. In recent years, a number of thinkers have argued that accelerating trends in computer science, cognitive science, and related disciplines foreshadow the creation of intelligent machines which meet and ultimately surpass the cognitive abilities of human beings, thereby entan…
▽ More
Characterizing human values is a topic deeply interwoven with the sciences, humanities, art, and many other human endeavors. In recent years, a number of thinkers have argued that accelerating trends in computer science, cognitive science, and related disciplines foreshadow the creation of intelligent machines which meet and ultimately surpass the cognitive abilities of human beings, thereby entangling an understanding of human values with future technological development. Contemporary research accomplishments suggest sophisticated AI systems becoming widespread and responsible for managing many aspects of the modern world, from preemptively planning users' travel schedules and logistics, to fully autonomous vehicles, to domestic robots assisting in daily living. The extrapolation of these trends has been most forcefully described in the context of a hypothetical "intelligence explosion," in which the capabilities of an intelligent software agent would rapidly increase due to the presence of feedback loops unavailable to biological organisms. The possibility of superintelligent agents, or simply the widespread deployment of sophisticated, autonomous AI systems, highlights an important theoretical problem: the need to separate the cognitive and rational capacities of an agent from the fundamental goal structure, or value system, which constrains and guides the agent's actions. The "value alignment problem" is to specify a goal structure for autonomous agents compatible with human values. In this brief article, we suggest that recent ideas from affective neuroscience and related disciplines aimed at characterizing neurological and behavioral universals in the mammalian class provide important conceptual foundations relevant to describing human values. We argue that the notion of "mammalian value systems" points to a potential avenue for fundamental research in AI safety and AI ethics.
△ Less
Submitted 21 January, 2019; v1 submitted 27 July, 2016;
originally announced July 2016.
-
A Statistical Model for Stroke Outcome Prediction and Treatment Planning
Authors:
Abhishek Sengupta,
Vaibhav Rajan,
Sakyajit Bhattacharya,
G R K Sarma
Abstract:
Stroke is a major cause of mortality and long--term disability in the world. Predictive outcome models in stroke are valuable for personalized treatment, rehabilitation planning and in controlled clinical trials. In this paper we design a new model to predict outcome in the short-term, the putative therapeutic window for several treatments. Our regression-based model has a parametric form that is…
▽ More
Stroke is a major cause of mortality and long--term disability in the world. Predictive outcome models in stroke are valuable for personalized treatment, rehabilitation planning and in controlled clinical trials. In this paper we design a new model to predict outcome in the short-term, the putative therapeutic window for several treatments. Our regression-based model has a parametric form that is designed to address many challenges common in medical datasets like highly correlated variables and class imbalance. Empirically our model outperforms the best--known previous models in predicting short--term outcomes and in inferring the most effective treatments that improve outcome.
△ Less
Submitted 22 February, 2016;
originally announced February 2016.
-
Unit Testing, Model Validation, and Biological Simulation
Authors:
Gopal P. Sarma,
Travis W. Jacobs,
Mark D. Watts,
Vahid Ghayoomi,
Richard C. Gerkin,
Stephen D. Larson
Abstract:
The growth of the software industry has gone hand in hand with the development of tools and cultural practices for ensuring the reliability of complex pieces of software. These tools and practices are now acknowledged to be essential to the management of modern software. As computational models and methods have become increasingly common in the biological sciences, it is important to examine how t…
▽ More
The growth of the software industry has gone hand in hand with the development of tools and cultural practices for ensuring the reliability of complex pieces of software. These tools and practices are now acknowledged to be essential to the management of modern software. As computational models and methods have become increasingly common in the biological sciences, it is important to examine how these practices can accelerate biological software development and improve research quality. In this article, we give a focused case study of our experience with the practices of unit testing and test-driven development in OpenWorm, an open-science project aimed at modeling Caenorhabditis elegans. We identify and discuss the challenges of incorporating test-driven development into a heterogeneous, data-driven project, as well as the role of model validation tests, a category of tests unique to software which expresses scientific models.
△ Less
Submitted 5 March, 2017; v1 submitted 19 August, 2015;
originally announced August 2015.
-
Should We Train Scientific Generalists?
Authors:
Gopal Sarma
Abstract:
I examine the topic of training scientific generalists. To focus the discussion, I propose the creation of a new graduate program, analogous in structure to existing MD/PhD programs, aimed at training a critical mass of scientific researchers with substantial intellectual breadth. In addition to completing the normal requirements for a PhD, students would undergo an intense, several year training…
▽ More
I examine the topic of training scientific generalists. To focus the discussion, I propose the creation of a new graduate program, analogous in structure to existing MD/PhD programs, aimed at training a critical mass of scientific researchers with substantial intellectual breadth. In addition to completing the normal requirements for a PhD, students would undergo an intense, several year training period designed to expose them to the core vocabulary of multiple subjects at the graduate level. After providing some historical and philosophical context for this proposal, I outline how such a program could be implemented with little institutional overhead by existing research universities. Finally, I discuss alternative possibilities for training generalists by taking advantage of contemporary developments in online learning and open science.
△ Less
Submitted 30 June, 2017; v1 submitted 11 October, 2014;
originally announced October 2014.
