-
Active Learning with Dual Model Predictive Path-Integral Control for Interaction-Aware Autonomous Highway On-ramp Merging
Authors:
Jacob Knaup,
Jovin D'sa,
Behdad Chalaki,
Tyler Naes,
Hossein Nourkhiz Mahjoub,
Ehsan Moradi-Pari,
Panagiotis Tsiotras
Abstract:
Merging into dense highway traffic for an autonomous vehicle is a complex decision-making task, wherein the vehicle must identify a potential gap and coordinate with surrounding human drivers, each of whom may exhibit diverse driving behaviors. Many existing methods consider other drivers to be dynamic obstacles and, as a result, are incapable of capturing the full intent of the human drivers via…
▽ More
Merging into dense highway traffic for an autonomous vehicle is a complex decision-making task, wherein the vehicle must identify a potential gap and coordinate with surrounding human drivers, each of whom may exhibit diverse driving behaviors. Many existing methods consider other drivers to be dynamic obstacles and, as a result, are incapable of capturing the full intent of the human drivers via this passive planning. In this paper, we propose a novel dual control framework based on Model Predictive Path-Integral control to generate interactive trajectories. This framework incorporates a Bayesian inference approach to actively learn the agents' parameters, i.e., other drivers' model parameters. The proposed framework employs a sampling-based approach that is suitable for real-time implementation through the utilization of GPUs. We illustrate the effectiveness of our proposed methodology through comprehensive numerical simulations conducted in both high and low-fidelity simulation scenarios focusing on autonomous on-ramp merging.
△ Less
Submitted 11 October, 2023;
originally announced October 2023.
-
Multi-Robot Cooperative Navigation in Crowds: A Game-Theoretic Learning-Based Model Predictive Control Approach
Authors:
Viet-Anh Le,
Vaishnav Tadiparthi,
Behdad Chalaki,
Hossein Nourkhiz Mahjoub,
Jovin D'sa,
Ehsan Moradi-Pari,
Andreas A. Malikopoulos
Abstract:
In this paper, we develop a control framework for the coordination of multiple robots as they navigate through crowded environments. Our framework comprises of a local model predictive control (MPC) for each robot and a social long short-term memory model that forecasts pedestrians' trajectories. We formulate the local MPC formulation for each individual robot that includes both individual and sha…
▽ More
In this paper, we develop a control framework for the coordination of multiple robots as they navigate through crowded environments. Our framework comprises of a local model predictive control (MPC) for each robot and a social long short-term memory model that forecasts pedestrians' trajectories. We formulate the local MPC formulation for each individual robot that includes both individual and shared objectives, in which the latter encourages the emergence of coordination among robots. Next, we consider the multi-robot navigation and human-robot interaction, respectively, as a potential game and a two-player game, then employ an iterative best response approach to solve the resulting optimization problems in a centralized and distributed fashion. Finally, we demonstrate the effectiveness of coordination among robots in simulated crowd navigation.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
Social Navigation in Crowded Environments with Model Predictive Control and Deep Learning-Based Human Trajectory Prediction
Authors:
Viet-Anh Le,
Behdad Chalaki,
Vaishnav Tadiparthi,
Hossein Nourkhiz Mahjoub,
Jovin D'sa,
Ehsan Moradi-Pari
Abstract:
Crowd navigation has received increasing attention from researchers over the last few decades, resulting in the emergence of numerous approaches aimed at addressing this problem to date. Our proposed approach couples agent motion prediction and planning to avoid the freezing robot problem while simultaneously capturing multi-agent social interactions by utilizing a state-of-the-art trajectory pred…
▽ More
Crowd navigation has received increasing attention from researchers over the last few decades, resulting in the emergence of numerous approaches aimed at addressing this problem to date. Our proposed approach couples agent motion prediction and planning to avoid the freezing robot problem while simultaneously capturing multi-agent social interactions by utilizing a state-of-the-art trajectory prediction model i.e., social long short-term memory model (Social-LSTM). Leveraging the output of Social-LSTM for the prediction of future trajectories of pedestrians at each time-step given the robot's possible actions, our framework computes the optimal control action using Model Predictive Control (MPC) for the robot to navigate among pedestrians. We demonstrate the effectiveness of our proposed approach in multiple scenarios of simulated crowd navigation and compare it against several state-of-the-art reinforcement learning-based methods.
△ Less
Submitted 28 September, 2023;
originally announced September 2023.
-
MR-IDM -- Merge Reactive Intelligent Driver Model: Towards Enhancing Laterally Aware Car-following Models
Authors:
Dustin Holley,
Jovin D'sa,
Hossein Nourkhiz Mahjoub,
Gibran Ali,
Behdad Chalaki,
Ehsan Moradi-Pari
Abstract:
This paper discusses the limitations of existing microscopic traffic models in accounting for the potential impacts of on-ramp vehicles on the car-following behavior of main-lane vehicles on highways. We first surveyed U.S. on-ramps to choose a representative set of on-ramps and then collected real-world observational data from the merging vehicle's perspective in various traffic conditions rangin…
▽ More
This paper discusses the limitations of existing microscopic traffic models in accounting for the potential impacts of on-ramp vehicles on the car-following behavior of main-lane vehicles on highways. We first surveyed U.S. on-ramps to choose a representative set of on-ramps and then collected real-world observational data from the merging vehicle's perspective in various traffic conditions ranging from free-flowing to rush-hour traffic jams. Next, as our core contribution, we introduce a novel car-following model, called MR-IDM, for highway driving that reacts to merging vehicles in a realistic way. This proposed driving model can either be used in traffic simulators to generate realistic highway driving behavior or integrated into a prediction module for autonomous vehicles attempting to merge onto the highway. We quantitatively evaluated the effectiveness of our model and compared it against several other methods. We show that MR-IDM has the least error in mimicking the real-world data, while having features such as smoothness, stability, and lateral awareness.
△ Less
Submitted 19 May, 2023;
originally announced May 2023.
-
Minimally Disruptive Cooperative Lane-change Maneuvers
Authors:
Behdad Chalaki,
Vaishnav Tadiparthi,
Hossein Nourkhiz Mahjoub,
Jovin D'sa,
Ehsan Moradi-Pari,
Andres S. Chavez Armijos,
Anni Li,
Christos G. Cassandras
Abstract:
A lane-change maneuver on a congested highway could be severely disruptive or even infeasible without the cooperation of neighboring cars. However, cooperation with other vehicles does not guarantee that the performed maneuver will not have a negative impact on traffic flow unless it is explicitly considered in the cooperative controller design. In this letter, we present a socially compliant fram…
▽ More
A lane-change maneuver on a congested highway could be severely disruptive or even infeasible without the cooperation of neighboring cars. However, cooperation with other vehicles does not guarantee that the performed maneuver will not have a negative impact on traffic flow unless it is explicitly considered in the cooperative controller design. In this letter, we present a socially compliant framework for cooperative lane-change maneuvers for an arbitrary number of CAVs on highways that aims to interrupt traffic flow as minimally as possible. Moreover, we explicitly impose feasibility constraints in the optimization formulation by using reachability set theory, leading to a unified design that removes the need for an iterative procedure used in prior work. We quantitatively evaluate the effectiveness of our framework and compare it against previously offered approaches in terms of maneuver time and incurred throughput disruption.
△ Less
Submitted 10 March, 2023;
originally announced March 2023.
-
Cooperative Energy and Time-Optimal Lane Change Maneuvers with Minimal Highway Traffic Disruption
Authors:
Andres S. Chavez Armijos,
Anni Li,
Christos G. Cassandras,
Yasir K. Al-Nadawi,
Hidekazu Araki,
Behdad Chalaki,
Ehsan Moradi-Pari,
Hossein Nourkhiz Mahjoub,
Vaishnav Tadiparthi
Abstract:
We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" only seeks to minimize…
▽ More
We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" only seeks to minimize its maneuver time, we seek to ensure that the fast-lane traffic flow is minimally disrupted (through a properly defined metric). Additionally, when performing lane-changing maneuvers, we optimally select the cooperating vehicles from a set of feasible neighboring vehicles and experimentally show that the highway throughput is improved compared to the baseline case of human-driven vehicles changing lanes with no cooperation. When feasible solutions do not exist for a given maximal allowable disruption, we include a time relaxation method trading off a longer maneuver time with reduced disruption. Our analysis is also extended to multiple sequential maneuvers. Simulation results show the effectiveness of our controllers in terms of safety guarantees and up to 16% and 90% average throughput and maneuver time improvement respectively when compared to maneuvers with no cooperation.
△ Less
Submitted 15 November, 2022;
originally announced November 2022.
-
A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways
Authors:
Sai Krishna Sumanth Nakka,
Behdad Chalaki,
Andreas Malikopoulos
Abstract:
The steady increase in the number of vehicles operating on the highways continues to exacerbate congestion, accidents, energy consumption, and greenhouse gas emissions. Emerging mobility systems, e.g., connected and automated vehicles (CAVs), have the potential to directly address these issues and improve transportation network efficiency and safety. In this paper, we consider a highway merging sc…
▽ More
The steady increase in the number of vehicles operating on the highways continues to exacerbate congestion, accidents, energy consumption, and greenhouse gas emissions. Emerging mobility systems, e.g., connected and automated vehicles (CAVs), have the potential to directly address these issues and improve transportation network efficiency and safety. In this paper, we consider a highway merging scenario and propose a framework for coordinating CAVs such that stop-and-go driving is eliminated. We use a decentralized form of the actor-critic approach to deep reinforcement learning$-$multi-agent deep deterministic policy gradient. We demonstrate the coordination of CAVs through numerical simulations and show that a smooth traffic flow is achieved by eliminating stop-and-go driving. Videos and plots of the simulation results can be found at this supplemental $\href{https://sites.google.com/view/ud-ids-lab/MADRL}{\text{site}}$.
△ Less
Submitted 13 March, 2022; v1 submitted 23 September, 2021;
originally announced September 2021.
-
A Scalable Last-Mile Delivery Service: From Simulation to Scaled Experiment
Authors:
Meera Ratnagiri,
Clare O'Dwyer,
Logan E. Beaver,
Heeseung Bang,
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
In this paper, we investigate the problem of a last-mile delivery service that selects up to $N$ available vehicles to deliver $M$ packages from a centralized depot to $M$ delivery locations. The objective of the last-mile delivery service is to jointly maximize customer satisfaction (minimize delivery time) and minimize operating cost (minimize total travel time) by selecting the optimal number o…
▽ More
In this paper, we investigate the problem of a last-mile delivery service that selects up to $N$ available vehicles to deliver $M$ packages from a centralized depot to $M$ delivery locations. The objective of the last-mile delivery service is to jointly maximize customer satisfaction (minimize delivery time) and minimize operating cost (minimize total travel time) by selecting the optimal number of vehicles to perform the deliveries. We model this as an assignment (vehicles to packages) and path planning (determining the delivery order and route) problem, which is equivalent to the NP-hard multiple traveling salesperson problem. We propose a scalable heuristic algorithm, which sacrifices some optimality to achieve a reasonable computational cost for a high number of packages. The algorithm combines hierarchical clustering with a greedy search. To validate our approach, we compare the results of our simulation to experiments in a $1$:$25$ scale robotic testbed for future mobility systems.
△ Less
Submitted 13 September, 2021;
originally announced September 2021.
-
A Digital Smart City for Emerging Mobility Systems
Authors:
Raymond M. Zayas,
Logan E. Beaver,
Behdad Chalaki,
Heeseung Bang,
Andreas A. Malikopoulos
Abstract:
The increasing demand for emerging mobility systems with connected and automated vehicles has imposed the necessity for quality testing environments to support their development. In this paper, we introduce a Unity-based virtual simulation environment for emerging mobility systems, called the Information and Decision Science Lab's Scaled Smart Digital City (IDS 3D City), intended to operate alongs…
▽ More
The increasing demand for emerging mobility systems with connected and automated vehicles has imposed the necessity for quality testing environments to support their development. In this paper, we introduce a Unity-based virtual simulation environment for emerging mobility systems, called the Information and Decision Science Lab's Scaled Smart Digital City (IDS 3D City), intended to operate alongside its physical peer and its established control framework. By utilizing the Robot Operation System, AirSim, and Unity, we constructed a simulation environment capable of iteratively designing experiments significantly faster than it is possible in a physical testbed. This environment provides an intermediate step to validate the effectiveness of our control algorithms prior to their implementation in the physical testbed. The IDS 3D City also enables us to demonstrate that our control algorithms work independently of the underlying vehicle dynamics, as the vehicle dynamics introduced by AirSim operate at a different scale than our scaled smart city. Finally, we demonstrate the behavior of our digital environment by performing an experiment in both the virtual and physical environments and comparing their outputs.
△ Less
Submitted 11 January, 2023; v1 submitted 6 September, 2021;
originally announced September 2021.
-
A Hysteretic Q-learning Coordination Framework for Emerging Mobility Systems in Smart Cities
Authors:
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
Connected and automated vehicles (CAVs) can alleviate traffic congestion, air pollution, and improve safety. In this paper, we provide a decentralized coordination framework for CAVs at a signal-free intersection to minimize travel time and improve fuel efficiency. We employ a simple yet powerful reinforcement learning approach, an off-policy temporal difference learning called Q-learning, enhance…
▽ More
Connected and automated vehicles (CAVs) can alleviate traffic congestion, air pollution, and improve safety. In this paper, we provide a decentralized coordination framework for CAVs at a signal-free intersection to minimize travel time and improve fuel efficiency. We employ a simple yet powerful reinforcement learning approach, an off-policy temporal difference learning called Q-learning, enhanced with a coordination mechanism to address this problem. Then, we integrate a first-in-first-out queuing policy to improve the performance of our system. We demonstrate the efficacy of our proposed approach through simulation and comparison with the classical optimal control method based on Pontryagin's minimum principle.
△ Less
Submitted 5 November, 2020;
originally announced November 2020.
-
Experimental Validation of a Real-Time Optimal Controller for Coordination of CAVs in a Multi-Lane Roundabout
Authors:
Behdad Chalaki,
Logan E. Beaver,
Andreas A. Malikopoulos
Abstract:
Roundabouts in conjunction with other traffic scenarios, e.g., intersections, merging roadways, speed reduction zones, can induce congestion in a transportation network due to driver responses to various disturbances. Research efforts have shown that smoothing traffic flow and eliminating stop-and-go driving can both improve fuel efficiency of the vehicles and the throughput of a roundabout. In th…
▽ More
Roundabouts in conjunction with other traffic scenarios, e.g., intersections, merging roadways, speed reduction zones, can induce congestion in a transportation network due to driver responses to various disturbances. Research efforts have shown that smoothing traffic flow and eliminating stop-and-go driving can both improve fuel efficiency of the vehicles and the throughput of a roundabout. In this paper, we validate an optimal control framework developed earlier in a multi-lane roundabout scenario using the University of Delaware's scaled smart city (UDSSC). We first provide conditions where the solution is optimal. Then, we demonstrate the feasibility of the solution using experiments at UDSSC, and show that the optimal solution completely eliminates stop-and-go driving while preserving safety.
△ Less
Submitted 18 May, 2020; v1 submitted 29 January, 2020;
originally announced January 2020.
-
Time-Optimal Coordination for Connected and Automated Vehicles at Adjacent Intersections
Authors:
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
In this paper, we provide a hierarchical coordination framework for connected and automated vehicles (CAVs) at two adjacent intersections. This framework consists of an upper-level scheduling problem and a low-level optimal control problem. By partitioning the area around two adjacent intersections into different zones, we formulate a scheduling problem for each individual CAV aimed at minimizing…
▽ More
In this paper, we provide a hierarchical coordination framework for connected and automated vehicles (CAVs) at two adjacent intersections. This framework consists of an upper-level scheduling problem and a low-level optimal control problem. By partitioning the area around two adjacent intersections into different zones, we formulate a scheduling problem for each individual CAV aimed at minimizing its total travel time. For each CAV, the solution of the upper-level problem designates the arrival times at each zones on its path which becomes the inputs of the low-level problem. The solution of the low-level problem yields the optimal control input (acceleration/deceleration) of each CAV to exit the intersections at the time specified in the upper-level scheduling problem. We validate the performance of our proposed hierarchical framework through extensive numerical simulations and comparison with signalized intersections, centralized scheduling, and FIFO queuing policy.
△ Less
Submitted 24 October, 2021; v1 submitted 11 November, 2019;
originally announced November 2019.
-
Demonstration of a Time-Efficient Mobility System Using a Scaled Smart City
Authors:
Logan E. Beaver,
Behdad Chalaki,
AM Ishtiaque Mahbub,
Liuhui Zhao,
Ray Zayas,
Andreas A. Malikopoulos
Abstract:
The implementation of connected and automated vehicle (CAV) technologies enables a novel computational framework to deliver real-time control actions that optimize travel time, energy, and safety. Hardware is an integral part of any practical implementation of CAVs, and as such, it should be incorporated in any validation method. However, high costs associated with full scale, field testing of CAV…
▽ More
The implementation of connected and automated vehicle (CAV) technologies enables a novel computational framework to deliver real-time control actions that optimize travel time, energy, and safety. Hardware is an integral part of any practical implementation of CAVs, and as such, it should be incorporated in any validation method. However, high costs associated with full scale, field testing of CAVs have proven to be a significant barrier. In this paper, we present the implementation of a decentralized control framework, which was developed previously, in a scaled-city using robotic CAVs, and discuss the implications of CAVs on travel time. Supplemental information and videos can be found at https://sites.google.com/view/ud-ids-lab/tfms.
△ Less
Submitted 21 November, 2019; v1 submitted 4 March, 2019;
originally announced March 2019.
-
Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles
Authors:
Kathy Jang,
Eugene Vinitsky,
Behdad Chalaki,
Ben Remer,
Logan Beaver,
Andreas Malikopoulos,
Alexandre Bayen
Abstract:
Using deep reinforcement learning, we train control policies for autonomous vehicles leading a platoon of vehicles onto a roundabout. Using Flow, a library for deep reinforcement learning in micro-simulators, we train two policies, one policy with noise injected into the state and action space and one without any injected noise. In simulation, the autonomous vehicle learns an emergent metering beh…
▽ More
Using deep reinforcement learning, we train control policies for autonomous vehicles leading a platoon of vehicles onto a roundabout. Using Flow, a library for deep reinforcement learning in micro-simulators, we train two policies, one policy with noise injected into the state and action space and one without any injected noise. In simulation, the autonomous vehicle learns an emergent metering behavior for both policies in which it slows to allow for smoother merging. We then directly transfer this policy without any tuning to the University of Delaware Scaled Smart City (UDSSC), a 1:25 scale testbed for connected and automated vehicles. We characterize the performance of both policies on the scaled city. We show that the noise-free policy winds up crashing and only occasionally metering. However, the noise-injected policy consistently performs the metering behavior and remains collision-free, suggesting that the noise helps with the zero-shot policy transfer. Additionally, the transferred, noise-injected policy leads to a 5% reduction of average travel time and a reduction of 22% in maximum travel time in the UDSSC. Videos of the controllers can be found at https://sites.google.com/view/iccps-policy-transfer.
△ Less
Submitted 22 February, 2019; v1 submitted 14 December, 2018;
originally announced December 2018.
