-
How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies
Authors:
Lukas M. Schmidt,
Sebastian Rietsch,
Axel Plinge,
Bjoern M. Eskofier,
Christopher Mutschler
Abstract:
Autonomous driving has the potential to revolutionize mobility and is hence an active area of research. In practice, the behavior of autonomous vehicles must be acceptable, i.e., efficient, safe, and interpretable. While vanilla reinforcement learning (RL) finds performant behavioral strategies, they are often unsafe and uninterpretable. Safety is introduced through Safe RL approaches, but they st…
▽ More
Autonomous driving has the potential to revolutionize mobility and is hence an active area of research. In practice, the behavior of autonomous vehicles must be acceptable, i.e., efficient, safe, and interpretable. While vanilla reinforcement learning (RL) finds performant behavioral strategies, they are often unsafe and uninterpretable. Safety is introduced through Safe RL approaches, but they still mostly remain uninterpretable as the learned behaviour is jointly optimized for safety and performance without modeling them separately. Interpretable machine learning is rarely applied to RL. This paper proposes SafeDQN, which allows to make the behavior of autonomous vehicles safe and interpretable while still being efficient. SafeDQN offers an understandable, semantic trade-off between the expected risk and the utility of actions while being algorithmically transparent. We show that SafeDQN finds interpretable and safe driving policies for a variety of scenarios and demonstrate how state-of-the-art saliency techniques can help to assess both risk and utility.
△ Less
Submitted 2 August, 2022; v1 submitted 16 March, 2022;
originally announced March 2022.
-
An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility
Authors:
Lukas M. Schmidt,
Johanna Brosig,
Axel Plinge,
Bjoern M. Eskofier,
Christopher Mutschler
Abstract:
Many scenarios in mobility and traffic involve multiple different agents that need to cooperate to find a joint solution. Recent advances in behavioral planning use Reinforcement Learning to find effective and performant behavior strategies. However, as autonomous vehicles and vehicle-to-X communications become more mature, solutions that only utilize single, independent agents leave potential per…
▽ More
Many scenarios in mobility and traffic involve multiple different agents that need to cooperate to find a joint solution. Recent advances in behavioral planning use Reinforcement Learning to find effective and performant behavior strategies. However, as autonomous vehicles and vehicle-to-X communications become more mature, solutions that only utilize single, independent agents leave potential performance gains on the road. Multi-Agent Reinforcement Learning (MARL) is a research field that aims to find optimal solutions for multiple agents that interact with each other. This work aims to give an overview of the field to researchers in autonomous mobility. We first explain MARL and introduce important concepts. Then, we discuss the central paradigms that underlie MARL algorithms, and give an overview of state-of-the-art methods and ideas in each paradigm. With this background, we survey applications of MARL in autonomous mobility scenarios and give an overview of existing scenarios and implementations.
△ Less
Submitted 2 August, 2022; v1 submitted 15 March, 2022;
originally announced March 2022.
-
The Exoplanet Transmission Spectroscopy Imager (ETSI)
Authors:
Mary Anne Limbach,
Luke M. Schmidt,
D. L. DePoy,
Jeffrey C. Mason,
Mike Scobey,
Pat Brown,
Chelsea Taylor,
Jennifer L. Marshall
Abstract:
We present the design of a novel instrument tuned to detect transiting exoplanet atmospheres. The instrument, which we call the exoplanet transmission spectroscopy imager (ETSI), makes use of a new technique called common-path multi-band imaging (CMI). ETSI uses a prism and multi-band filter to simultaneously image 15 spectral bandpasses on two detectors from $430-975nm$ (with a average spectral r…
▽ More
We present the design of a novel instrument tuned to detect transiting exoplanet atmospheres. The instrument, which we call the exoplanet transmission spectroscopy imager (ETSI), makes use of a new technique called common-path multi-band imaging (CMI). ETSI uses a prism and multi-band filter to simultaneously image 15 spectral bandpasses on two detectors from $430-975nm$ (with a average spectral resolution of $R = λ/Δλ= 23$) during exoplanet transits of a bright star. A prototype of the instrument achieved photon-noise limited results which were below the atmospheric amplitude scintillation noise limit. ETSI can detect the presence and composition of an exoplanet atmosphere in a relatively short time on a modest-size telescope. We show the optical design of the instrument. Further, we discuss design trades of the prism and multi-band filter which are driven by the science of the ETSI instrument. We describe the upcoming survey with ETSI that will measure dozens of exoplanet atmosphere spectra in $\sim2$ years on a two meter telescope. Finally, we will discuss how ETSI will be a powerful means for follow up on all gas giant exoplanets that transit bright stars, including a multitude of recently identified TESS (NASA's Transiting Exoplanet Survey Satellite) exoplanets.
△ Less
Submitted 1 December, 2020;
originally announced December 2020.
