Showing 1–7 of 7 results for author: Nice, M

Reinforcement Learning Based Oscillation Dampening: Scaling up Single-Agent RL algorithms to a 100 AV highway field operational test

Authors: Kathy Jang, Nathan Lichtlé, Eugene Vinitsky, Adit Shah, Matthew Bunting, Matthew Nice, Benedetto Piccoli, Benjamin Seibold, Daniel B. Work, Maria Laura Delle Monache, Jonathan Sprinkle, Jonathan W. Lee, Alexandre M. Bayen

Abstract: In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with develo** RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their app… ▽ More In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with develo** RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function sha**. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. We explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles. △ Less

Submitted 14 May, 2024; v1 submitted 26 February, 2024; originally announced February 2024.

A Middle Way to Traffic Enlightenment

Authors: Matthew W. Nice, George Gunter, Junyi Ji, Yuhang Zhang, Matthew Bunting, Will Barbour, Jonathan Sprinkle, Dan Work

Abstract: This paper introduces a novel approach that seeks a middle ground for traffic control in multi-lane congestion, where prevailing traffic speeds are too fast, and speed recommendations designed to dampen traffic waves are too slow. Advanced controllers that modify the speed of an automated car for wave-dampening, eco-driving, or other goals, typically are designed with forward collision safety in m… ▽ More This paper introduces a novel approach that seeks a middle ground for traffic control in multi-lane congestion, where prevailing traffic speeds are too fast, and speed recommendations designed to dampen traffic waves are too slow. Advanced controllers that modify the speed of an automated car for wave-dampening, eco-driving, or other goals, typically are designed with forward collision safety in mind. Our approach goes further, by considering how dangerous it can be for a controller to drive so slowly relative to prevailing traffic that it creates a significant issue for safety and comfort. This paper explores open-road scenarios where large gaps between prevailing speeds and desired speeds can exist, specifically when infrastructure-based variable speed limit systems are not strictly followed at all times by other drivers. Our designed, implemented, and deployed algorithm is able to follow variable speed limits when others also follow it, avoid collisions with vehicles ahead, and adapt to prevailing traffic when other motorists are traveling well above the posted speeds. The key is to reject unsafe speed recommendations from infrastructure-based traffic smoothing systems, based on real-time local traffic conditions observed by the vehicle under control. This solution is implemented and deployed on two control vehicles in heavy multi-lane highway congestion. The results include analysis from system design, and field tests that validate the system's performance using an existing Variable Speed Limit system as the external source for speed recommendations, and the on-board sensors of a stock Toyota Rav4 for inputs that estimate the prevailing speed of traffic around the vehicle under control. △ Less

Submitted 28 January, 2024; originally announced January 2024.

Comments: To Appear, ICCPS 2024

Enabling Mixed Autonomy Traffic Control

Authors: Matthew Nice, Matt Bunting, Alex Richardson, Gergely Zachar, Jonathan W. Lee, Alexandre Bayen, Maria Laura Delle Monache, Benjamin Seibold, Benedetto Piccoli, Jonathan Sprinkle, Dan Work

Abstract: We demonstrate a new capability of automated vehicles: mixed autonomy traffic control. With this new capability, automated vehicles can shape the traffic flows composed of other non-automated vehicles, which has the promise to improve safety, efficiency, and energy outcomes in transportation systems at a societal scale. Investigating mixed autonomy mobile traffic control must be done in situ given… ▽ More We demonstrate a new capability of automated vehicles: mixed autonomy traffic control. With this new capability, automated vehicles can shape the traffic flows composed of other non-automated vehicles, which has the promise to improve safety, efficiency, and energy outcomes in transportation systems at a societal scale. Investigating mixed autonomy mobile traffic control must be done in situ given that the complex dynamics of other drivers and their response to a team of automated vehicles cannot be effectively modeled. This capability has been blocked because there is no existing scalable and affordable platform for experimental control. This paper introduces an extensible open-source hardware and software platform, enabling a team of 100 vehicles to execute several different vehicular control algorithms as a collaborative fleet, composed of three different makes and models, which drove 22752 miles in a combined 1022 hours, over 5 days in Nashville, TN in November 2022. △ Less

Submitted 28 October, 2023; originally announced October 2023.

SAILing CAVs: Speed-Adaptive Infrastructure-Linked Connected and Automated Vehicles

Authors: Matthew Nice, Matthew Bunting, George Gunter, William Barbour, Jonathan Sprinkle, Dan Work

Abstract: This work demonstrates a new capability in roadway control: Speed-adaptive, infrastructure-linked connected and automated vehicles. We develop and deploy a lightly modified vehicle that is able to dynamically adjust the vehicle speed in response to posted variable speed limit messages generated by the infrastructure using LTE connectivity. This work describes the open source hardware and software… ▽ More This work demonstrates a new capability in roadway control: Speed-adaptive, infrastructure-linked connected and automated vehicles. We develop and deploy a lightly modified vehicle that is able to dynamically adjust the vehicle speed in response to posted variable speed limit messages generated by the infrastructure using LTE connectivity. This work describes the open source hardware and software platform that enables integration between infrastructure-based variable posted speed limits, and existing vehicle platforms for automated control. The vehicle is deployed in heavy morning traffic on I-24 in Nashville, TN. The control vehicle follows the posted variable speed limits, resulting in as much as a 25% reduction in speed variability compared to a human-piloted vehicle in the same traffic stream. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

So you think you can track?

Authors: Derek Gloudemans, Gergely Zachár, Yanbing Wang, Junyi Ji, Matt Nice, Matt Bunting, William Barbour, Jonathan Sprinkle, Benedetto Piccoli, Maria Laura Delle Monache, Alexandre Bayen, Benjamin Seibold, Daniel B. Work

Abstract: This work introduces a multi-camera tracking dataset consisting of 234 hours of video data recorded concurrently from 234 overlap** HD cameras covering a 4.2 mile stretch of 8-10 lane interstate highway near Nashville, TN. The video is recorded during a period of high traffic density with 500+ objects typically visible within the scene and typical object longevities of 3-15 minutes. GPS trajecto… ▽ More This work introduces a multi-camera tracking dataset consisting of 234 hours of video data recorded concurrently from 234 overlap** HD cameras covering a 4.2 mile stretch of 8-10 lane interstate highway near Nashville, TN. The video is recorded during a period of high traffic density with 500+ objects typically visible within the scene and typical object longevities of 3-15 minutes. GPS trajectories from 270 vehicle passes through the scene are manually corrected in the video data to provide a set of ground-truth trajectories for recall-oriented tracking metrics, and object detections are provided for each camera in the scene (159 million total before cross-camera fusion). Initial benchmarking of tracking-by-detection algorithms is performed against the GPS trajectories, and a best HOTA of only 9.5% is obtained (best recall 75.9% at IOU 0.1, 47.9 average IDs per ground truth object), indicating the benchmarked trackers do not perform sufficiently well at the long temporal and spatial durations required for traffic scene understanding. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Compromised ACC vehicles can degrade current mixed-autonomy traffic performance while remaining stealthy against detection

Authors: George Gunter, Huichen Li, Avesta Hojjati, Matthew Nice, Matthew Bunting, Carl A. Gunter, Bo Li, Jonathan Sprinkle, Daniel Work

Abstract: We demonstrate that a supply-chain level compromise of the adaptive cruise control (ACC) capability on equipped vehicles can be used to significantly degrade system level performance of current day mixed-autonomy freeway networks. Via a simple threat model which causes random deceleration attacks (RDAs), compromised vehicles create congestion waves in the traffic that decrease average speed and ne… ▽ More We demonstrate that a supply-chain level compromise of the adaptive cruise control (ACC) capability on equipped vehicles can be used to significantly degrade system level performance of current day mixed-autonomy freeway networks. Via a simple threat model which causes random deceleration attacks (RDAs), compromised vehicles create congestion waves in the traffic that decrease average speed and network throughput. We use a detailed and realistic traffic simulation environment to quantify the impacts of the attack on a model of a real high-volume freeway in the United States. We find that the effect of the attack depends both on the level of underlying traffic congestion, and what percentage of ACC vehicles can be compromised. In moderate congestion regimes the attack can degrade mean commuter speed by over 7%. In high density regimes overall network throughput can be reduced by up to 3%. And, in moderate to high congestion regimes, it can cost commuters on the network over 300 USD/km hr. All of these results motivate that the proposed attack is able to significantly degrade performance of the traffic network. We also develop an anomaly detection technique that uses GPS traces on vehicles to identify malicious/compromised vehicles. We employ this technique on data from the simulation experiments and find that it is unable to identify compromised ACCs compared to benign/normal drivers. That is, these attacks are stealthy to detection. Stronger attacks can be accurately labeled as malicious, motivating that there is a limit to how impactful attacks can be before they are no longer stealthy. Finally, we experimentally execute the attack on a real and commercially available ACC vehicle, demonstrating the possible real world feasibility of an RDA. △ Less

Submitted 22 December, 2021; originally announced December 2021.

CAN Coach: Vehicular Control through Human Cyber-Physical Systems

Authors: M. Nice, S. Elmadani, R. Bhadani, M. Bunting, J. Sprinkle, D. Work

Abstract: This work addresses whether a human-in-the-loop cyber-physical system (HCPS) can be effective in improving the longitudinal control of an individual vehicle in a traffic flow. We introduce the CAN Coach, which is a system that gives feedback to the human-in-the-loop using radar data (relative speed and position information to objects ahead) that is available on the controller area network (CAN). U… ▽ More This work addresses whether a human-in-the-loop cyber-physical system (HCPS) can be effective in improving the longitudinal control of an individual vehicle in a traffic flow. We introduce the CAN Coach, which is a system that gives feedback to the human-in-the-loop using radar data (relative speed and position information to objects ahead) that is available on the controller area network (CAN). Using a cohort of six human subjects driving an instrumented vehicle, we compare the ability of the human-in-the-loop driver to achieve a constant time-gap control policy using only human-based visual perception to the car ahead, and by augmenting human perception with audible feedback from CAN sensor data. The addition of CAN-based feedback reduces the mean time-gap error by an average of 73%, and also improves the consistency of the human by reducing the standard deviation of the time-gap error by 53%. We remove human perception from the loop using a ghost mode in which the human-in-the-loop is coached to track a virtual vehicle on the road, rather than a physical one. The loss of visual perception of the vehicle ahead degrades the performance for most drivers, but by varying amounts. We show that human subjects can match the velocity of the lead vehicle ahead with and without CAN-based feedback, but velocity matching does not offer regulation of vehicle spacing. The viability of dynamic time-gap control is also demonstrated. We conclude that (1) it is possible to coach drivers to improve performance on driving tasks using CAN data, and (2) it is a true HCPS, since removing human perception from the control loop reduces performance at the given control objective. △ Less

Submitted 8 April, 2021; originally announced April 2021.