-
Explanation through Reward Model Reconciliation using POMDP Tree Search
Authors:
Benjamin D. Kraske,
Anshu Saksena,
Anna L. Buczak,
Zachary N. Sunberg
Abstract:
As artificial intelligence (AI) algorithms are increasingly used in mission-critical applications, promoting user-trust of these systems will be essential to their success. Ensuring users understand the models over which algorithms reason promotes user trust. This work seeks to reconcile differences between the reward model that an algorithm uses for online partially observable Markov decision (PO…
▽ More
As artificial intelligence (AI) algorithms are increasingly used in mission-critical applications, promoting user-trust of these systems will be essential to their success. Ensuring users understand the models over which algorithms reason promotes user trust. This work seeks to reconcile differences between the reward model that an algorithm uses for online partially observable Markov decision (POMDP) planning and the implicit reward model assumed by a human user. Action discrepancies, differences in decisions made by an algorithm and user, are leveraged to estimate a user's objectives as expressed in weightings of a reward function.
△ Less
Submitted 1 May, 2023;
originally announced May 2023.
-
A Generative Adversarial Network for Climate Tip** Point Discovery (TIP-GAN)
Authors:
Jennifer Sleeman,
David Chung,
Anand Gnanadesikan,
Jay Brett,
Yannis Kevrekidis,
Marisa Hughes,
Thomas Haine,
Marie-Aude Pradal,
Renske Gelderloos,
Chace Ashcraft,
Caroline Tang,
Anshu Saksena,
Larry White
Abstract:
We propose a new Tip** Point Generative Adversarial Network (TIP-GAN) for better characterizing potential climate tip** points in Earth system models. We describe an adversarial game to explore the parameter space of these models, detect upcoming tip** points, and discover the drivers of tip** points. In this setup, a set of generators learn to construct model configurations that will invo…
▽ More
We propose a new Tip** Point Generative Adversarial Network (TIP-GAN) for better characterizing potential climate tip** points in Earth system models. We describe an adversarial game to explore the parameter space of these models, detect upcoming tip** points, and discover the drivers of tip** points. In this setup, a set of generators learn to construct model configurations that will invoke a climate tip** point. The discriminator learns to identify which generators are generating each model configuration and whether a given configuration will lead to a tip** point. The discriminator is trained using an oracle (a surrogate climate model) to test if a generated model configuration leads to a tip** point or not. We demonstrate the application of this GAN to invoke the collapse of the Atlantic Meridional Overturning Circulation (AMOC). We share experimental results of modifying the loss functions and the number of generators to exploit the area of uncertainty in model state space near a climate tip** point. In addition, we show that our trained discriminator can predict AMOC collapse with a high degree of accuracy without the use of the oracle. This approach could generalize to other tip** points, and could augment climate modeling research by directing users interested in studying tip** points to parameter sets likely to induce said tip** points in their computationally intensive climate models.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
Using Artificial Intelligence to aid Scientific Discovery of Climate Tip** Points
Authors:
Jennifer Sleeman,
David Chung,
Chace Ashcraft,
Jay Brett,
Anand Gnanadesikan,
Yannis Kevrekidis,
Marisa Hughes,
Thomas Haine,
Marie-Aude Pradal,
Renske Gelderloos,
Caroline Tang,
Anshu Saksena,
Larry White
Abstract:
We propose a hybrid Artificial Intelligence (AI) climate modeling approach that enables climate modelers in scientific discovery using a climate-targeted simulation methodology based on a novel combination of deep neural networks and mathematical methods for modeling dynamical systems. The simulations are grounded by a neuro-symbolic language that both enables question answering of what is learned…
▽ More
We propose a hybrid Artificial Intelligence (AI) climate modeling approach that enables climate modelers in scientific discovery using a climate-targeted simulation methodology based on a novel combination of deep neural networks and mathematical methods for modeling dynamical systems. The simulations are grounded by a neuro-symbolic language that both enables question answering of what is learned by the AI methods and provides a means of explainability. We describe how this methodology can be applied to the discovery of climate tip** points and, in particular, the collapse of the Atlantic Meridional Overturning Circulation (AMOC). We show how this methodology is able to predict AMOC collapse with a high degree of accuracy using a surrogate climate model for ocean interaction. We also show preliminary results of neuro-symbolic method performance when translating between natural language questions and symbolically learned representations. Our AI methodology shows promising early results, potentially enabling faster climate tip** point related research that would otherwise be computationally infeasible.
△ Less
Submitted 14 February, 2023;
originally announced February 2023.
-
Towards Indirect Top-Down Road Transport Emissions Estimation
Authors:
Ryan Mukherjee,
Derek Rollend,
Gordon Christie,
Armin Hadzic,
Sally Matson,
Anshu Saksena,
Marisa Hughes
Abstract:
Road transportation is one of the largest sectors of greenhouse gas (GHG) emissions affecting climate change. Tackling climate change as a global community will require new capabilities to measure and inventory road transport emissions. However, the large scale and distributed nature of vehicle emissions make this sector especially challenging for existing inventory methods. In this work, we devel…
▽ More
Road transportation is one of the largest sectors of greenhouse gas (GHG) emissions affecting climate change. Tackling climate change as a global community will require new capabilities to measure and inventory road transport emissions. However, the large scale and distributed nature of vehicle emissions make this sector especially challenging for existing inventory methods. In this work, we develop machine learning models that use satellite imagery to perform indirect top-down estimation of road transport emissions. Our initial experiments focus on the United States, where a bottom-up inventory was available for training our models. We achieved a mean absolute error (MAE) of 39.5 kg CO$_{2}$ of annual road transport emissions, calculated on a pixel-by-pixel (100 m$^{2}$) basis in Sentinel-2 imagery. We also discuss key model assumptions and challenges that need to be addressed to develop models capable of generalizing to global geography. We believe this work is the first published approach for automated indirect top-down estimation of road transport sector emissions using visual imagery and represents a critical step towards scalable, global, near-real-time road transportation emissions inventories that are measured both independently and objectively.
△ Less
Submitted 15 March, 2021;
originally announced March 2021.
-
Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning
Authors:
Ritchie Lee,
Ole J. Mengshoel,
Anshu Saksena,
Ryan Gardner,
Daniel Genin,
Joshua Silbermann,
Michael Owen,
Mykel J. Kochenderfer
Abstract:
Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, saf…
▽ More
Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present formulations for fully observable and partially observable systems. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.
△ Less
Submitted 4 December, 2020; v1 submitted 6 November, 2018;
originally announced November 2018.
