Predictive representations: building blocks of intelligence
Authors:
Wilka Carvalho,
Momchil S. Tomov,
William de Cothi,
Caswell Barry,
Samuel J. Gershman
Abstract:
Adaptive behavior often requires predicting future events. The theory of reinforcement learning prescribes what kinds of predictive representations are useful and how to compute them. This paper integrates these theoretical ideas with work on cognition and neuroscience. We pay special attention to the successor representation (SR) and its generalizations, which have been widely applied both as eng…
▽ More
Adaptive behavior often requires predicting future events. The theory of reinforcement learning prescribes what kinds of predictive representations are useful and how to compute them. This paper integrates these theoretical ideas with work on cognition and neuroscience. We pay special attention to the successor representation (SR) and its generalizations, which have been widely applied both as engineering tools and models of brain function. This convergence suggests that particular kinds of predictive representations may function as versatile building blocks of intelligence.
△ Less
Submitted 17 April, 2024; v1 submitted 9 February, 2024;
originally announced February 2024.
Driving in Real Life with Inverse Reinforcement Learning
Authors:
Tung Phan-Minh,
Forbes Howington,
Ting-Sheng Chu,
Sang Uk Lee,
Momchil S. Tomov,
Nanxiang Li,
Caglayan Dicle,
Samuel Findler,
Francisco Suarez-Ruiz,
Robert Beaudoin,
Bo Yang,
Sammy Omari,
Eric M. Wolff
Abstract:
In this paper, we introduce the first learning-based planner to drive a car in dense, urban traffic using Inverse Reinforcement Learning (IRL). Our planner, DriveIRL, generates a diverse set of trajectory proposals, filters these trajectories with a lightweight and interpretable safety filter, and then uses a learned model to score each remaining trajectory. The best trajectory is then tracked by…
▽ More
In this paper, we introduce the first learning-based planner to drive a car in dense, urban traffic using Inverse Reinforcement Learning (IRL). Our planner, DriveIRL, generates a diverse set of trajectory proposals, filters these trajectories with a lightweight and interpretable safety filter, and then uses a learned model to score each remaining trajectory. The best trajectory is then tracked by the low-level controller of our self-driving vehicle. We train our trajectory scoring model on a 500+ hour real-world dataset of expert driving demonstrations in Las Vegas within the maximum entropy IRL framework. DriveIRL's benefits include: a simple design due to only learning the trajectory scoring function, relatively interpretable features, and strong real-world performance. We validated DriveIRL on the Las Vegas Strip and demonstrated fully autonomous driving in heavy traffic, including scenarios involving cut-ins, abrupt braking by the lead vehicle, and hotel pickup/dropoff zones. Our dataset will be made public to help further research in this area.
△ Less
Submitted 7 June, 2022;
originally announced June 2022.
