-
Scalable Learning of Segment-Level Traffic Congestion Functions
Authors:
Shushman Choudhury,
Abdul Rahman Kreidieh,
Iveel Tsogsuren,
Neha Arora,
Carolina Osorio,
Alexandre Bayen
Abstract:
We propose and study a data-driven framework for identifying traffic congestion functions (numerical relationships between observations of macroscopic traffic variables) at global scale and segment-level granularity. In contrast to methods that estimate a separate set of parameters for each roadway, ours learns a single black-box function over all roadways in a metropolitan area. First, we pool tr…
▽ More
We propose and study a data-driven framework for identifying traffic congestion functions (numerical relationships between observations of macroscopic traffic variables) at global scale and segment-level granularity. In contrast to methods that estimate a separate set of parameters for each roadway, ours learns a single black-box function over all roadways in a metropolitan area. First, we pool traffic data from all segments into one dataset, combining static attributes with dynamic time-dependent features. Second, we train a feed-forward neural network on this dataset, which we can then use on any segment in the area. We evaluate how well our framework identifies congestion functions on observed segments and how it generalizes to unobserved segments and predicts segment attributes on a large dataset covering multiple cities worldwide. For identification error on observed segments, our single data-driven congestion function compares favorably to segment-specific model-based functions on highway roads, but has room to improve on arterial roads. For generalization, our approach shows strong performance across cities and road types: both on unobserved segments in the same city and on zero-shot transfer learning between cities. Finally, for predicting segment attributes, we find that our approach can approximate critical densities for individual segments using their static properties.
△ Less
Submitted 9 May, 2024;
originally announced May 2024.
-
Traffic Control via Connected and Automated Vehicles: An Open-Road Field Experiment with 100 CAVs
Authors:
Jonathan W. Lee,
Han Wang,
Kathy Jang,
Amaury Hayat,
Matthew Bunting,
Arwa Alanqary,
William Barbour,
Zhe Fu,
Xiaoqian Gong,
George Gunter,
Sharon Hornstein,
Abdul Rahman Kreidieh,
Nathan Lichtlé,
Matthew W. Nice,
William A. Richardson,
Adit Shah,
Eugene Vinitsky,
Fangyu Wu,
Shengquan Xiang,
Sulaiman Almatrudi,
Fahd Althukair,
Rahul Bhadani,
Joy Carpio,
Raphael Chekroun,
Eric Cheng
, et al. (39 additional authors not shown)
Abstract:
The CIRCLES project aims to reduce instabilities in traffic flow, which are naturally occurring phenomena due to human driving behavior. These "phantom jams" or "stop-and-go waves,"are a significant source of wasted energy. Toward this goal, the CIRCLES project designed a control system referred to as the MegaController by the CIRCLES team, that could be deployed in real traffic. Our field experim…
▽ More
The CIRCLES project aims to reduce instabilities in traffic flow, which are naturally occurring phenomena due to human driving behavior. These "phantom jams" or "stop-and-go waves,"are a significant source of wasted energy. Toward this goal, the CIRCLES project designed a control system referred to as the MegaController by the CIRCLES team, that could be deployed in real traffic. Our field experiment leveraged a heterogeneous fleet of 100 longitudinally-controlled vehicles as Lagrangian traffic actuators, each of which ran a controller with the architecture described in this paper. The MegaController is a hierarchical control architecture, which consists of two main layers. The upper layer is called Speed Planner, and is a centralized optimal control algorithm. It assigns speed targets to the vehicles, conveyed through the LTE cellular network. The lower layer is a control layer, running on each vehicle. It performs local actuation by overriding the stock adaptive cruise controller, using the stock on-board sensors. The Speed Planner ingests live data feeds provided by third parties, as well as data from our own control vehicles, and uses both to perform the speed assignment. The architecture of the speed planner allows for modular use of standard control techniques, such as optimal control, model predictive control, kernel methods and others, including Deep RL, model predictive control and explicit controllers. Depending on the vehicle architecture, all onboard sensing data can be accessed by the local controllers, or only some. Control inputs vary across different automakers, with inputs ranging from torque or acceleration requests for some cars, and electronic selection of ACC set points in others. The proposed architecture allows for the combination of all possible settings proposed above. Most configurations were tested throughout the ramp up to the MegaVandertest.
△ Less
Submitted 26 February, 2024;
originally announced February 2024.
-
Hierarchical Speed Planner for Automated Vehicles: A Framework for Lagrangian Variable Speed Limit in Mixed Autonomy Traffic
Authors:
Han Wang,
Zhe Fu,
Jonathan Lee,
Hossein Nick Zinat Matin,
Arwa Alanqary,
Daniel Urieli,
Sharon Hornstein,
Abdul Rahman Kreidieh,
Raphael Chekroun,
William Barbour,
William A. Richardson,
Dan Work,
Benedetto Piccoli,
Benjamin Seibold,
Jonathan Sprinkle,
Alexandre M. Bayen,
Maria Laura Delle Monache
Abstract:
This paper introduces a novel control framework for Lagrangian variable speed limits in hybrid traffic flow environments utilizing automated vehicles (AVs). The framework was validated using a fleet of 100 connected automated vehicles as part of the largest coordinated open-road test designed to smooth traffic flow. The framework includes two main components: a high-level controller deployed on th…
▽ More
This paper introduces a novel control framework for Lagrangian variable speed limits in hybrid traffic flow environments utilizing automated vehicles (AVs). The framework was validated using a fleet of 100 connected automated vehicles as part of the largest coordinated open-road test designed to smooth traffic flow. The framework includes two main components: a high-level controller deployed on the server side, named Speed Planner, and low-level controllers called vehicle controllers deployed on the vehicle side. The Speed Planner designs and updates target speeds for the vehicle controllers based on real-time Traffic State Estimation (TSE) [1]. The Speed Planner comprises two modules: a TSE enhancement module and a target speed design module. The TSE enhancement module is designed to minimize the effects of inherent latency in the received traffic information and to improve the spatial and temporal resolution of the input traffic data. The target speed design module generates target speed profiles with the goal of improving traffic flow. The vehicle controllers are designed to track the target speed meanwhile responding to the surrounding situation. The numerical simulation indicates the performance of the proposed method: the bottleneck throughput has increased by 5.01%, and the speed standard deviation has been reduced by a significant 34.36%. We further showcase an operational study with a description of how the controller was implemented on a field-test with 100 AVs and its comprehensive effects on the traffic flow.
△ Less
Submitted 26 February, 2024;
originally announced February 2024.
-
Cooperative Driving for Speed Harmonization in Mixed-Traffic Environments
Authors:
Zhe Fu,
Abdul Rahman Kreidieh,
Han Wang,
Jonathan W. Lee,
Maria Laura Delle Monache,
Alexandre M. Bayen
Abstract:
Autonomous driving systems present promising methods for congestion mitigation in mixed autonomy traffic control settings. In particular, when coupled with even modest traffic state estimates, such systems can plan and coordinate the behaviors of automated vehicles (AVs) in response to observed downstream events, thereby inhibiting the continued propagation of congestion. In this paper, we present…
▽ More
Autonomous driving systems present promising methods for congestion mitigation in mixed autonomy traffic control settings. In particular, when coupled with even modest traffic state estimates, such systems can plan and coordinate the behaviors of automated vehicles (AVs) in response to observed downstream events, thereby inhibiting the continued propagation of congestion. In this paper, we present a two-layer control strategy in which the upper layer proposes the desired speeds that predictively react to the downstream state of traffic, and the lower layer maintains safe and reasonable headways with leading vehicles. This method is demonstrated to achieve an average of over 15% energy savings within simulations of congested events observed in Interstate 24 with only 4% AV penetration, while restricting negative externalities imposed on traveling times and mobility. The proposed strategy that served as part of the "speed planner" was deployed on 100 AVs in a massive traffic experiment conducted on Nashville's I-24 in November 2022.
△ Less
Submitted 3 June, 2023; v1 submitted 14 February, 2023;
originally announced February 2023.
-
Learning energy-efficient driving behaviors by imitating experts
Authors:
Abdul Rahman Kreidieh,
Zhe Fu,
Alexandre M. Bayen
Abstract:
The rise of vehicle automation has generated significant interest in the potential role of future automated vehicles (AVs). In particular, in highly dense traffic settings, AVs are expected to serve as congestion-dampeners, mitigating the presence of instabilities that arise from various sources. However, in many applications, such maneuvers rely heavily on non-local sensing or coordination by int…
▽ More
The rise of vehicle automation has generated significant interest in the potential role of future automated vehicles (AVs). In particular, in highly dense traffic settings, AVs are expected to serve as congestion-dampeners, mitigating the presence of instabilities that arise from various sources. However, in many applications, such maneuvers rely heavily on non-local sensing or coordination by interacting AVs, thereby rendering their adaptation to real-world settings a particularly difficult challenge. To address this challenge, this paper examines the role of imitation learning in bridging the gap between such control strategies and realistic limitations in communication and sensing. Treating one such controller as an "expert", we demonstrate that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations. Results and code are available online at https://sites.google.com/view/il-traffic/home.
△ Less
Submitted 28 June, 2022;
originally announced August 2022.
-
Unified Automatic Control of Vehicular Systems with Reinforcement Learning
Authors:
Zhongxia Yan,
Abdul Rahman Kreidieh,
Eugene Vinitsky,
Alexandre M. Bayen,
Cathy Wu
Abstract:
Emerging vehicular systems with increasing proportions of automated components present opportunities for optimal control to mitigate congestion and increase efficiency. There has been a recent interest in applying deep reinforcement learning (DRL) to these nonlinear dynamical systems for the automatic design of effective control strategies. Despite conceptual advantages of DRL being model-free, st…
▽ More
Emerging vehicular systems with increasing proportions of automated components present opportunities for optimal control to mitigate congestion and increase efficiency. There has been a recent interest in applying deep reinforcement learning (DRL) to these nonlinear dynamical systems for the automatic design of effective control strategies. Despite conceptual advantages of DRL being model-free, studies typically nonetheless rely on training setups that are painstakingly specialized to specific vehicular systems. This is a key challenge to efficient analysis of diverse vehicular and mobility systems. To this end, this article contributes a streamlined methodology for vehicular microsimulation and discovers high performance control strategies with minimal manual design. A variable-agent, multi-task approach is presented for optimization of vehicular Partially Observed Markov Decision Processes. The methodology is experimentally validated on mixed autonomy traffic systems, where fractions of vehicles are automated; empirical improvement, typically 15-60% over a human driving baseline, is observed in all configurations of six diverse open or closed traffic systems. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering. Finally, the emergent behaviors are analyzed to produce interpretable control strategies, which are validated against the learned control strategies.
△ Less
Submitted 30 July, 2022;
originally announced August 2022.
-
Non-local Evasive Overtaking of Downstream Incidents in Distributed Behavior Planning of Connected Vehicles
Authors:
Abdul Rahman Kreidieh,
Yashar Farid,
Kentaro Oguchi
Abstract:
The prevalence of high-speed vehicle-to-everything (V2X) communication will likely significantly influence the future of vehicle autonomy. In several autonomous driving applications, however, the role such systems will play is seldom understood. In this paper, we explore the role of communication signals in enhancing the performance of lane change assistance systems in situations where downstream…
▽ More
The prevalence of high-speed vehicle-to-everything (V2X) communication will likely significantly influence the future of vehicle autonomy. In several autonomous driving applications, however, the role such systems will play is seldom understood. In this paper, we explore the role of communication signals in enhancing the performance of lane change assistance systems in situations where downstream bottlenecks restrict the mobility of a few lanes. Building off of prior work on modeling lane change incentives, we design a controller that 1) encourages automated vehicles to subvert lanes in which distant downstream delays are likely to occur, while also 2) ignoring greedy local incentives when such delays are needed to maintain a specific route. Numerical results on different traffic conditions and penetration rates suggest that the model successfully subverts a significant portion of delays brought about by downstream bottlenecks, both globally and from the perspective of the controlled vehicles.
△ Less
Submitted 29 June, 2022;
originally announced June 2022.
-
Learning Generalizable Multi-Lane Mixed-Autonomy Behaviors in Single Lane Representations of Traffic
Authors:
Abdul Rahman Kreidieh,
Yibo Zhao,
Samyak Parajuli,
Alexandre Bayen
Abstract:
Reinforcement learning techniques can provide substantial insights into the desired behaviors of future autonomous driving systems. By optimizing for societal metrics of traffic such as increased throughput and reduced energy consumption, such methods can derive maneuvers that, if adopted by even a small portion of vehicles, may significantly improve the state of traffic for all vehicles involved.…
▽ More
Reinforcement learning techniques can provide substantial insights into the desired behaviors of future autonomous driving systems. By optimizing for societal metrics of traffic such as increased throughput and reduced energy consumption, such methods can derive maneuvers that, if adopted by even a small portion of vehicles, may significantly improve the state of traffic for all vehicles involved. These methods, however, are hindered in practice by the difficulty of designing efficient and accurate models of traffic, as well as the challenges associated with optimizing for the behaviors of dozens of interacting agents. In response to these challenges, this paper tackles the problem of learning generalizable traffic control strategies in simple representations of vehicle driving dynamics. In particular, we look to mixed-autonomy ring roads as depictions of instabilities that result in the formation of congestion. Within this problem, we design a curriculum learning paradigm that exploits the natural extendability of the network to effectively learn behaviors that reduce congestion over long horizons. Next, we study the implications of modeling lane changing on the transferability of policies. Our findings suggest that introducing lane change behaviors that even approximately match trends in more complex systems can significantly improve the generalizability of subsequent learned models to more accurate multi-lane models of traffic.
△ Less
Submitted 31 December, 2021; v1 submitted 8 December, 2021;
originally announced December 2021.
-
Integrated Framework of Vehicle Dynamics, Instabilities, Energy Models, and Sparse Flow Smoothing Controllers
Authors:
Jonathan W. Lee,
George Gunter,
Rabie Ramadan,
Sulaiman Almatrudi,
Paige Arnold,
John Aquino,
William Barbour,
Rahul Bhadani,
Joy Carpio,
Fang-Chieh Chou,
Marsalis Gibson,
Xiaoqian Gong,
Amaury Hayat,
Nour Khoudari,
Abdul Rahman Kreidieh,
Maya Kumar,
Nathan Lichtlé,
Sean McQuade,
Brian Nguyen,
Megan Ross,
Sydney Truong,
Eugene Vinitsky,
Yibo Zhao,
Jonathan Sprinkle,
Benedetto Piccoli
, et al. (3 additional authors not shown)
Abstract:
This work presents an integrated framework of: vehicle dynamics models, with a particular attention to instabilities and traffic waves; vehicle energy models, with particular attention to accurate energy values for strongly unsteady driving profiles; and sparse Lagrangian controls via automated vehicles, with a focus on controls that can be executed via existing technology such as adaptive cruise…
▽ More
This work presents an integrated framework of: vehicle dynamics models, with a particular attention to instabilities and traffic waves; vehicle energy models, with particular attention to accurate energy values for strongly unsteady driving profiles; and sparse Lagrangian controls via automated vehicles, with a focus on controls that can be executed via existing technology such as adaptive cruise control systems. This framework serves as a key building block in develo** control strategies for human-in-the-loop traffic flow smoothing on real highways. In this contribution, we outline the fundamental merits of integrating vehicle dynamics and energy modeling into a single framework, and we demonstrate the energy impact of sparse flow smoothing controllers via simulation results.
△ Less
Submitted 22 April, 2021;
originally announced April 2021.
-
ResiliNet: Failure-Resilient Inference in Distributed Neural Networks
Authors:
Ashkan Yousefpour,
Brian Q. Nguyen,
Siddartha Devic,
Guanhua Wang,
Aboudy Kreidieh,
Hans Lobel,
Alexandre M. Bayen,
Jason P. Jue
Abstract:
Federated Learning aims to train distributed deep models without sharing the raw data with the centralized server. Similarly, in distributed inference of neural networks, by partitioning the network and distributing it across several physical nodes, activations and gradients are exchanged between physical nodes, rather than raw data. Nevertheless, when a neural network is partitioned and distribut…
▽ More
Federated Learning aims to train distributed deep models without sharing the raw data with the centralized server. Similarly, in distributed inference of neural networks, by partitioning the network and distributing it across several physical nodes, activations and gradients are exchanged between physical nodes, rather than raw data. Nevertheless, when a neural network is partitioned and distributed among physical nodes, failure of physical nodes causes the failure of the neural units that are placed on those nodes, which results in a significant performance drop. Current approaches focus on resiliency of training in distributed neural networks. However, resiliency of inference in distributed neural networks is less explored. We introduce ResiliNet, a scheme for making inference in distributed neural networks resilient to physical node failures. ResiliNet combines two concepts to provide resiliency: skip hyperconnection, a concept for skip** nodes in distributed neural networks similar to skip connection in resnets, and a novel technique called failout, which is introduced in this paper. Failout simulates physical node failure conditions during training using dropout, and is specifically designed to improve the resiliency of distributed neural networks. The results of the experiments and ablation studies using three datasets confirm the ability of ResiliNet to provide inference resiliency for distributed neural networks.
△ Less
Submitted 19 December, 2020; v1 submitted 18 February, 2020;
originally announced February 2020.
-
Inter-Level Cooperation in Hierarchical Reinforcement Learning
Authors:
Abdul Rahman Kreidieh,
Glen Berseth,
Brandon Trabucco,
Samyak Parajuli,
Sergey Levine,
Alexandre M. Bayen
Abstract:
Hierarchies of temporally decoupled policies present a promising approach for enabling structured exploration in complex long-term planning problems. To fully achieve this approach an end-to-end training paradigm is needed. However, training these multi-level policies has had limited success due to challenges arising from interactions between the goal-assigning and goal-achieving levels within a h…
▽ More
Hierarchies of temporally decoupled policies present a promising approach for enabling structured exploration in complex long-term planning problems. To fully achieve this approach an end-to-end training paradigm is needed. However, training these multi-level policies has had limited success due to challenges arising from interactions between the goal-assigning and goal-achieving levels within a hierarchy. In this article, we consider the policy optimization process as a multi-agent process. This allows us to draw on connections between communication and cooperation in multi-agent RL, and demonstrate the benefits of increased cooperation between sub-policies on the training performance of the overall policy. We introduce a simple yet effective technique for inducing inter-level cooperation by modifying the objective function and subsequent gradients of higher-level policies. Experimental results on a wide variety of simulated robotics and traffic control tasks demonstrate that inducing cooperation results in stronger performing policies and increased sample efficiency on a set of difficult long time horizon tasks. We also find that goal-conditioned policies trained using our method display better transfer to new tasks, highlighting the benefits of our method in learning task-agnostic lower-level behaviors. Videos and code are available at: https://sites.google.com/berkeley.edu/cooperative-hrl.
△ Less
Submitted 17 November, 2021; v1 submitted 4 December, 2019;
originally announced December 2019.
-
Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud
Authors:
Ashkan Yousefpour,
Siddartha Devic,
Brian Q. Nguyen,
Aboudy Kreidieh,
Alan Liao,
Alexandre M. Bayen,
Jason P. Jue
Abstract:
Partitioning and distributing deep neural networks (DNNs) over physical nodes such as edge, fog, or cloud nodes, could enhance sensor fusion, and reduce bandwidth and inference latency. However, when a DNN is distributed over physical nodes, failure of the physical nodes causes the failure of the DNN units that are placed on these nodes. The performance of the inference task will be unpredictable,…
▽ More
Partitioning and distributing deep neural networks (DNNs) over physical nodes such as edge, fog, or cloud nodes, could enhance sensor fusion, and reduce bandwidth and inference latency. However, when a DNN is distributed over physical nodes, failure of the physical nodes causes the failure of the DNN units that are placed on these nodes. The performance of the inference task will be unpredictable, and most likely, poor, if the distributed DNN is not specifically designed and properly trained for failures. Motivated by this, we introduce deepFogGuard, a DNN architecture augmentation scheme for making the distributed DNN inference task failure-resilient. To articulate deepFogGuard, we introduce the elements and a model for the resiliency of distributed DNN inference. Inspired by the concept of residual connections in DNNs, we introduce skip hyperconnections in distributed DNNs, which are the basis of deepFogGuard's design to provide resiliency. Next, our extensive experiments using two existing datasets for the sensing and vision applications confirm the ability of deepFogGuard to provide resiliency for distributed DNNs in edge-cloud networks.
△ Less
Submitted 21 September, 2019; v1 submitted 3 September, 2019;
originally announced September 2019.
-
Flow: A Modular Learning Framework for Mixed Autonomy Traffic
Authors:
Cathy Wu,
Aboudy Kreidieh,
Kanaad Parvate,
Eugene Vinitsky,
Alexandre M Bayen
Abstract:
The rapid development of autonomous vehicles (AVs) holds vast potential for transportation systems through improved safety, efficiency, and access to mobility. However, the progression of these impacts, as AVs are adopted, is not well understood. Numerous technical challenges arise from the goal of analyzing the partial adoption of autonomy: partial control and observation, multi-vehicle interacti…
▽ More
The rapid development of autonomous vehicles (AVs) holds vast potential for transportation systems through improved safety, efficiency, and access to mobility. However, the progression of these impacts, as AVs are adopted, is not well understood. Numerous technical challenges arise from the goal of analyzing the partial adoption of autonomy: partial control and observation, multi-vehicle interactions, and the sheer variety of scenarios represented by real-world networks. To shed light into near-term AV impacts, this article studies the suitability of deep reinforcement learning (RL) for overcoming these challenges in a low AV-adoption regime. A modular learning framework is presented, which leverages deep RL to address complex traffic dynamics. Modules are composed to capture common traffic phenomena (stop-and-go traffic jams, lane changing, intersections). Learned control laws are found to improve upon human driving performance, in terms of system-level velocity, by up to 57% with only 4-7% adoption of AVs. Furthermore, in single-lane traffic, a small neural network control law with only local observation is found to eliminate stop-and-go traffic - surpassing all known model-based controllers to achieve near-optimal performance - and generalize to out-of-distribution traffic densities.
△ Less
Submitted 30 December, 2021; v1 submitted 15 October, 2017;
originally announced October 2017.