-
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Authors:
David "davidad" Dalrymple,
Joar Skalse,
Yoshua Bengio,
Stuart Russell,
Max Tegmark,
Sanjit Seshia,
Steve Omohundro,
Christian Szegedy,
Ben Goldhaber,
Nora Ammann,
Alessandro Abate,
Joe Halpern,
Clark Barrett,
Ding Zhao,
Tan Zhi-Xuan,
Jeannette Wing,
Joshua Tenenbaum
Abstract:
Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these appro…
▽ More
Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. This is achieved by the interplay of three core components: a world model (which provides a mathematical description of how the AI system affects the outside world), a safety specification (which is a mathematical description of what effects are acceptable), and a verifier (which provides an auditable proof certificate that the AI satisfies the safety specification relative to the world model). We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them. We also argue for the necessity of this approach to AI safety, and for the inadequacy of the main alternative approaches.
△ Less
Submitted 8 July, 2024; v1 submitted 10 May, 2024;
originally announced May 2024.
-
Transforming Probabilistic Programs for Model Checking
Authors:
Ryan Bernstein,
Matthijs Vákár,
Jeannette Wing
Abstract:
Probabilistic programming is perfectly suited to reliable and transparent data science, as it allows the user to specify their models in a high-level language without worrying about the complexities of how to fit the models. Static analysis of probabilistic programs presents even further opportunities for enabling a high-level style of programming, by automating time-consuming and error-prone task…
▽ More
Probabilistic programming is perfectly suited to reliable and transparent data science, as it allows the user to specify their models in a high-level language without worrying about the complexities of how to fit the models. Static analysis of probabilistic programs presents even further opportunities for enabling a high-level style of programming, by automating time-consuming and error-prone tasks. We apply static analysis to probabilistic programs to automate large parts of two crucial model checking methods: Prior Predictive Checks and Simulation-Based Calibration. Our method transforms a probabilistic program specifying a density function into an efficient forward-sampling form. To achieve this transformation, we extract a factor graph from a probabilistic program using static analysis, generate a set of proposal directed acyclic graphs using a SAT solver, select a graph which will produce provably correct sampling code, then generate one or more sampling programs. We allow minimal user interaction to broaden the scope of application beyond what is possible with static analysis alone. We present an implementation targeting the popular Stan probabilistic programming language, automating large parts of a robust Bayesian workflow for a wide community of probabilistic programming users.
△ Less
Submitted 21 August, 2020;
originally announced August 2020.
-
Ensuring Fairness Beyond the Training Data
Authors:
Debmalya Mandal,
Samuel Deng,
Suman Jana,
Jeannette M. Wing,
Daniel Hsu
Abstract:
We initiate the study of fair classifiers that are robust to perturbations in the training distribution. Despite recent progress, the literature on fairness has largely ignored the design of fair and robust classifiers. In this work, we develop classifiers that are fair not only with respect to the training distribution, but also for a class of distributions that are weighted perturbations of the…
▽ More
We initiate the study of fair classifiers that are robust to perturbations in the training distribution. Despite recent progress, the literature on fairness has largely ignored the design of fair and robust classifiers. In this work, we develop classifiers that are fair not only with respect to the training distribution, but also for a class of distributions that are weighted perturbations of the training samples. We formulate a min-max objective function whose goal is to minimize a distributionally robust training loss, and at the same time, find a classifier that is fair with respect to a class of distributions. We first reduce this problem to finding a fair classifier that is robust with respect to the class of distributions. Based on online learning algorithm, we develop an iterative algorithm that provably converges to such a fair and robust solution. Experiments on standard machine learning fairness datasets suggest that, compared to the state-of-the-art fair classifiers, our classifier retains fairness guarantees and test accuracy for a large class of perturbations on the test set. Furthermore, our experiments show that there is an inherent trade-off between fairness robustness and accuracy of such classifiers.
△ Less
Submitted 4 November, 2020; v1 submitted 12 July, 2020;
originally announced July 2020.
-
Trustworthy AI
Authors:
Jeannette M. Wing
Abstract:
The promise of AI is huge. AI systems have already achieved good enough performance to be in our streets and in our homes. However, they can be brittle and unfair. For society to reap the benefits of AI systems, society needs to be able to trust them. Inspired by decades of progress in trustworthy computing, we suggest what trustworthy properties would be desired of AI systems. By enumerating a se…
▽ More
The promise of AI is huge. AI systems have already achieved good enough performance to be in our streets and in our homes. However, they can be brittle and unfair. For society to reap the benefits of AI systems, society needs to be able to trust them. Inspired by decades of progress in trustworthy computing, we suggest what trustworthy properties would be desired of AI systems. By enumerating a set of new research questions, we explore one approach--formal verification--for ensuring trust in AI. Trustworthy AI ups the ante on both trustworthy computing and formal methods.
△ Less
Submitted 14 February, 2020;
originally announced February 2020.
-
Ten Research Challenge Areas in Data Science
Authors:
Jeannette M. Wing
Abstract:
Although data science builds on knowledge from computer science, mathematics, statistics, and other disciplines, data science is a unique field with many mysteries to unlock: challenging scientific questions and pressing questions of societal importance. This article starts with meta-questions about data science as a discipline and then elaborates on ten ideas for the basis of a research agenda fo…
▽ More
Although data science builds on knowledge from computer science, mathematics, statistics, and other disciplines, data science is a unique field with many mysteries to unlock: challenging scientific questions and pressing questions of societal importance. This article starts with meta-questions about data science as a discipline and then elaborates on ten ideas for the basis of a research agenda for data science.
△ Less
Submitted 27 January, 2020;
originally announced February 2020.
-
The High-Redshift Clusters Occupied by Bent Radio AGN (COBRA) Survey: The \spitzer Catalog
Authors:
R. Paterno-Mahler,
E. L. Blanton,
M. Brodwin,
M. L. N. Ashby,
E. Golden-Marx,
B. Decker,
J. D. Wing,
G. Anand
Abstract:
We present 190 galaxy cluster candidates (most at high redshift) based on galaxy overdensity measurements in the \spitzer/IRAC imaging of the fields surrounding 646 bent, double-lobed radio sources drawn from the Clusters Occupied by Bent Radio AGN (COBRA) Survey. The COBRA sources were chosen as objects in the VLA FIRST survey that lack optical counterparts in the Sloan Digital Sky Survey (SDSS)…
▽ More
We present 190 galaxy cluster candidates (most at high redshift) based on galaxy overdensity measurements in the \spitzer/IRAC imaging of the fields surrounding 646 bent, double-lobed radio sources drawn from the Clusters Occupied by Bent Radio AGN (COBRA) Survey. The COBRA sources were chosen as objects in the VLA FIRST survey that lack optical counterparts in the Sloan Digital Sky Survey (SDSS) to a limit of $m_r=22$, making them likely to lie at high redshift. This is confirmed by our observations: the redshift distribution of COBRA sources with estimated redshifts peaks near $z=1$, and extends out to $z\approx3$. Cluster candidates were identified by comparing our target fields to a background field and searching for statistically significant ($\ge2σ$) excesses in the galaxy number counts surrounding the radio sources; 190 fields satisfy the $\ge2σ$ limit. We find that 530 fields (82.0\%) have a net positive excess of galaxies surrounding the radio source. Many of the fields with positive excesses but below the $2σ$ cutoff are likely to be galaxy groups. Forty-one COBRA sources are quasars with known spectroscopic redshifts, which may be tracers of some of the most distant clusters known.
△ Less
Submitted 22 June, 2017; v1 submitted 2 November, 2016;
originally announced November 2016.
-
Radio Galaxy Zoo: discovery of a poor cluster through a giant wide-angle tail radio galaxy
Authors:
J. K. Banfield,
H. Andernach,
A. D. Kapinska,
L. Rudnick,
M. J. Hardcastle,
G. Cotter,
S. Vaughan,
T. W. Jones,
I. Heywood,
J. D. Wing,
O. I. Wong,
T. Matorny,
I. A. Terentev,
A. R. Lopez-Sanchez,
R. P. Norris,
N. Seymour,
S. S. Shabala,
K. W. Willett
Abstract:
We have discovered a previously unreported poor cluster of galaxies (RGZ-CL J0823.2+0333) through an unusual giant wide-angle tail radio galaxy found in the Radio Galaxy Zoo project. We obtained a spectroscopic redshift of $z=0.0897$ for the E0-type host galaxy, 2MASX J08231289+0333016, leading to M$_r = -22.6$ and a $1.4\,$GHz radio luminosity density of $L_{\rm 1.4} = 5.5\times10^{24}$ W Hz…
▽ More
We have discovered a previously unreported poor cluster of galaxies (RGZ-CL J0823.2+0333) through an unusual giant wide-angle tail radio galaxy found in the Radio Galaxy Zoo project. We obtained a spectroscopic redshift of $z=0.0897$ for the E0-type host galaxy, 2MASX J08231289+0333016, leading to M$_r = -22.6$ and a $1.4\,$GHz radio luminosity density of $L_{\rm 1.4} = 5.5\times10^{24}$ W Hz$^{-1}$. These radio and optical luminosities are typical for wide-angle tailed radio galaxies near the borderline between Fanaroff-Riley (FR) classes I and II. The projected largest angular size of $\approx8\,$arcmin corresponds to $800\,$kpc and the full length of the source along the curved jets/trails is $1.1\,$Mpc in projection. X-ray data from the XMM-Newton archive yield an upper limit on the X-ray luminosity of the thermal emission surrounding RGZ J082312.9+033301,at $1.2-2.6\times10^{43}$ erg s$^{-1}$ for assumed intra-cluster medium temperatures of $1.0-5.0\,$keV. Our analysis of the environment surrounding RGZ J082312.9+033301 indicates that RGZ J082312.9+033301 lies within a poor cluster. The observed radio morphology suggests that (a) the host galaxy is moving at a significant velocity with respect to an ambient medium like that of at least a poor cluster, and that (b) the source may have had two ignition events of the active galactic nucleus with $10^7\,$yrs in between. This reinforces the idea that an association between RGZ J082312.9+033301, and the newly discovered poor cluster exists.
△ Less
Submitted 16 June, 2016; v1 submitted 15 June, 2016;
originally announced June 2016.
-
Inverse Privacy
Authors:
Yuri Gurevich,
Efim Hudis,
Jeannette M. Wing
Abstract:
An item of your personal information is inversely private if some party has access to it but you do not. We analyze the provenance of inversely private information and its rise to dominance over other kinds of personal information. In a nutshell, the inverse privacy problem is unjustified inaccessibility to you of your inversely private information. We argue that the inverse privacy problem has a…
▽ More
An item of your personal information is inversely private if some party has access to it but you do not. We analyze the provenance of inversely private information and its rise to dominance over other kinds of personal information. In a nutshell, the inverse privacy problem is unjustified inaccessibility to you of your inversely private information. We argue that the inverse privacy problem has a market-based solution.
△ Less
Submitted 12 October, 2015;
originally announced October 2015.
-
Extragalactic Jets as Probes of Distant Clusters of Galaxies and the Clusters Occupied by Bent Radio AGN (COBRA) Survey
Authors:
Elizabeth L. Blanton,
Rachel Paterno-Mahler,
Joshua D. Wing,
M. L. N. Ashby,
Emmet Golden-Marx,
Mark Brodwin,
E. M. Douglass,
Scott W. Randall,
T. E. Clarke
Abstract:
We are conducting a large survey of distant clusters of galaxies using radio sources with bent jets and lobes as tracers. These radio sources are driven by AGN and achieve their bent morphologies through interaction with the surrounding gas found in clusters of galaxies. Based on low-redshift studies, these types of sources can be used to identify clusters very efficiently. We present initial resu…
▽ More
We are conducting a large survey of distant clusters of galaxies using radio sources with bent jets and lobes as tracers. These radio sources are driven by AGN and achieve their bent morphologies through interaction with the surrounding gas found in clusters of galaxies. Based on low-redshift studies, these types of sources can be used to identify clusters very efficiently. We present initial results from our survey of 653 bent-double radio sources with optical hosts too faint to appear in the SDSS. The sample was observed in the infrared with Spitzer, and it has revealed $\sim$200 distant clusters or proto-clusters in the redshift range $z\sim0.7 - 3.0$. The sample of bent-doubles contains both quasars and radio galaxies enabling us to study both radiative and kinetic mode feedback in cluster and group environments at a wide range of redshifts.
△ Less
Submitted 21 November, 2014;
originally announced November 2014.
-
A Methodology for Information Flow Experiments
Authors:
Michael Carl Tschantz,
Amit Datta,
Anupam Datta,
Jeannette M. Wing
Abstract:
Information flow analysis has largely ignored the setting where the analyst has neither control over nor a complete model of the analyzed system. We formalize such limited information flow analyses and study an instance of it: detecting the usage of data by websites. We prove that these problems are ones of causal inference. Leveraging this connection, we push beyond traditional information flow a…
▽ More
Information flow analysis has largely ignored the setting where the analyst has neither control over nor a complete model of the analyzed system. We formalize such limited information flow analyses and study an instance of it: detecting the usage of data by websites. We prove that these problems are ones of causal inference. Leveraging this connection, we push beyond traditional information flow analysis to provide a systematic methodology based on experimental science and statistical analysis. Our methodology allows us to systematize prior works in the area viewing them as instances of a general approach. Our systematic study leads to practical advice for improving work on detecting data usage, a previously unformalized area. We illustrate these concepts with a series of experiments collecting data on the use of information by websites, which we statistically analyze.
△ Less
Submitted 9 May, 2014;
originally announced May 2014.
-
An Examination of the Optical Substructure of Galaxy Clusters Hosting Radio Sources
Authors:
Joshua D. Wing,
Elizabeth L. Blanton
Abstract:
Using radio sources from the Faint Images of the Radio Sky at Twenty-cm (FIRST) survey, and optical counterparts in the Sloan Digital Sky Survey (SDSS), we have identified a large number of galaxy clusters. The radio sources within these clusters are driven by active galactic nuclei, and our cluster samples include clusters with bent, and straight, double-lobed radio sources. We also included a si…
▽ More
Using radio sources from the Faint Images of the Radio Sky at Twenty-cm (FIRST) survey, and optical counterparts in the Sloan Digital Sky Survey (SDSS), we have identified a large number of galaxy clusters. The radio sources within these clusters are driven by active galactic nuclei, and our cluster samples include clusters with bent, and straight, double-lobed radio sources. We also included a single-radio-component comparison sample. We examine these galaxy clusters for evidence of optical substructure, testing the possibility that bent double-lobed radio sources are formed as a result of large-scale cluster mergers. We use a suite of substructure analysis tools to determine the location and extent of substructure visible in the optical distribution of cluster galaxies, and compare the rates of substructure in clusters with different types of radio sources. We found no preference for significant substructure in clusters hosting bent double-lobed radio sources compared to those with other types of radio sources.
△ Less
Submitted 1 May, 2013; v1 submitted 14 November, 2012;
originally announced November 2012.
-
On the Semantics of Purpose Requirements in Privacy Policies
Authors:
Michael Carl Tschantz,
Anupam Datta,
Jeannette M. Wing
Abstract:
Privacy policies often place requirements on the purposes for which a governed entity may use personal information. For example, regulations, such as HIPAA, require that hospital employees use medical information for only certain purposes, such as treatment. Thus, using formal or automated methods for enforcing privacy policies requires a semantics of purpose requirements to determine whether an a…
▽ More
Privacy policies often place requirements on the purposes for which a governed entity may use personal information. For example, regulations, such as HIPAA, require that hospital employees use medical information for only certain purposes, such as treatment. Thus, using formal or automated methods for enforcing privacy policies requires a semantics of purpose requirements to determine whether an action is for a purpose or not. We provide such a semantics using a formalism based on planning. We model planning using a modified version of Markov Decision Processes, which exclude redundant actions for a formal definition of redundant. We use the model to formalize when a sequence of actions is only for or not for a purpose. This semantics enables us to provide an algorithm for automating auditing, and to describe formally and compare rigorously previous enforcement methods.
△ Less
Submitted 21 February, 2011;
originally announced February 2011.
-
The Merger Environment of the WAT Hosting Cluster Abell 562
Authors:
E. M. Douglass,
Elizabeth L. Blanton,
T. E. Clarke,
Scott W. Randall,
Joshua D. Wing
Abstract:
We present a Chandra X-ray observation and VLA radio observations of the nearby (z=0.11) galaxy cluster Abell 562 and the wide angle tail (WAT) radio source 0647+693. The cluster displays signatures of an ongoing merger leading to the bending of the WAT source including an elongation of the X-ray surface brightness distribution along the line that bisects the WAT, an excess of displaced gas found…
▽ More
We present a Chandra X-ray observation and VLA radio observations of the nearby (z=0.11) galaxy cluster Abell 562 and the wide angle tail (WAT) radio source 0647+693. The cluster displays signatures of an ongoing merger leading to the bending of the WAT source including an elongation of the X-ray surface brightness distribution along the line that bisects the WAT, an excess of displaced gas found between the radio lobes, and anisotropies within the ICM projected temperature and abundance distributions. The most likely geometry of the ongoing interaction is a head-on merger occurring along the WAT bending axis. By combining observable properties of A562 and 0647+693 with common values for the conditions within merging clusters at the time of core crossing, we constrain the internal density (rho[ j ] = 0.001 rho[ICM]) of the jets and plasma flow velocity within the lobes (v = 0.02c - 0.03c) of the WAT source.
△ Less
Submitted 22 October, 2010; v1 submitted 20 October, 2010;
originally announced October 2010.
-
Galaxy Cluster Environments of Radio Sources
Authors:
Joshua D. Wing,
Elizabeth L. Blanton
Abstract:
Using the Sloan Digital Sky Survey (SDSS) and the FIRST (Faint Images of the Radio Sky at Twenty Centimeters) catalogs, we examined the optical environments around double-lobed radio sources. Previous studies have shown that multi-component radio sources exhibiting some degree of bending between components are likely to be found in galaxy clusters. Often this radio emission is associated with a cD…
▽ More
Using the Sloan Digital Sky Survey (SDSS) and the FIRST (Faint Images of the Radio Sky at Twenty Centimeters) catalogs, we examined the optical environments around double-lobed radio sources. Previous studies have shown that multi-component radio sources exhibiting some degree of bending between components are likely to be found in galaxy clusters. Often this radio emission is associated with a cD-type galaxy at the center of a cluster. We cross-correlated the SDSS and FIRST catalogs and measured the richness of the cluster environments surrounding both bent and straight multi-component radio sources. This led to the discovery and classification of a large number of galaxy clusters out to a redshift of z ~ 0.5. We divided our sample into smaller subgroups based on their optical and radio properties. We find that FR I radio sources are more likely to be found in galaxy clusters than FR II sources. Further, we find that bent radio sources are more often found in galaxy clusters than non-bent radio sources. We also examined the environments around single-component radio sources and find that single-component radio sources are less likely to be associated with galaxy clusters than extended, multi-component radio sources. Bent, visually-selected sources are found in clusters or rich groups ~78% of the time. Those without optical hosts in SDSS are likely associated with clusters at even higher redshifts, most with redshifts of z > 0.7.
△ Less
Submitted 6 January, 2011; v1 submitted 5 August, 2010;
originally announced August 2010.
-
On Bus Graph Realizability
Authors:
Anil Ada,
Melanie Coggan,
Paul Di Marco,
Alain Doyon,
Liam Flookes,
Samuli Heilala,
Ethan Kim,
Jonathan Li On Wing,
Louis-Francois Preville-Ratelle,
Sue Whitesides,
Nuo Yu
Abstract:
In this paper, we consider the following graph embedding problem: Given a bipartite graph G = (V1; V2;E), where the maximum degree of vertices in V2 is 4, can G be embedded on a two dimensional grid such that each vertex in V1 is drawn as a line segment along a grid line, each vertex in V2 is drawn as a point at a grid point, and each edge e = (u; v) for some u 2 V1 and v 2 V2 is drawn as a line…
▽ More
In this paper, we consider the following graph embedding problem: Given a bipartite graph G = (V1; V2;E), where the maximum degree of vertices in V2 is 4, can G be embedded on a two dimensional grid such that each vertex in V1 is drawn as a line segment along a grid line, each vertex in V2 is drawn as a point at a grid point, and each edge e = (u; v) for some u 2 V1 and v 2 V2 is drawn as a line segment connecting u and v, perpendicular to the line segment for u? We show that this problem is NP-complete, and sketch how our proof techniques can be used to show the hardness of several other related problems.
△ Less
Submitted 22 September, 2006;
originally announced September 2006.
