-
Data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics
Authors:
C. Chen,
Y. P. Huang,
W. H. K. Lam,
T. L. Pan,
S. C. Hsu,
A. Sumalee,
R. X. Zhong
Abstract:
Existing data-driven and feedback traffic control strategies do not consider the heterogeneity of real-time data measurements. Besides, traditional reinforcement learning (RL) methods for traffic control usually converge slowly for lacking data efficiency. Moreover, conventional optimal perimeter control schemes require exact knowledge of the system dynamics and thus would be fragile to endogenous…
▽ More
Existing data-driven and feedback traffic control strategies do not consider the heterogeneity of real-time data measurements. Besides, traditional reinforcement learning (RL) methods for traffic control usually converge slowly for lacking data efficiency. Moreover, conventional optimal perimeter control schemes require exact knowledge of the system dynamics and thus would be fragile to endogenous uncertainties. To handle these challenges, this work proposes an integral reinforcement learning (IRL) based approach to learning the macroscopic traffic dynamics for adaptive optimal perimeter control. This work makes the following primary contributions to the transportation literature: (a) A continuous-time control is developed with discrete gain updates to adapt to the discrete-time sensor data. (b) To reduce the sampling complexity and use the available data more efficiently, the experience replay (ER) technique is introduced to the IRL algorithm. (c) The proposed method relaxes the requirement on model calibration in a "model-free" manner that enables robustness against modeling uncertainty and enhances the real-time performance via a data-driven RL algorithm. (d) The convergence of the IRL-based algorithms and the stability of the controlled traffic dynamics are proven via the Lyapunov theory. The optimal control law is parameterized and then approximated by neural networks (NN), which moderates the computational complexity. Both state and input constraints are considered while no model linearization is required. Numerical examples and simulation experiments are presented to verify the effectiveness and efficiency of the proposed method.
△ Less
Submitted 13 September, 2022;
originally announced September 2022.
-
Domain Adversarial Spatial-Temporal Network: A Transferable Framework for Short-term Traffic Forecasting across Cities
Authors:
Yihong Tang,
Ao Qu,
Andy H. F. Chow,
William H. K. Lam,
S. C. Wong,
Wei Ma
Abstract:
Accurate real-time traffic forecast is critical for intelligent transportation systems (ITS) and it serves as the cornerstone of various smart mobility applications. Though this research area is dominated by deep learning, recent studies indicate that the accuracy improvement by develo** new model structures is becoming marginal. Instead, we envision that the improvement can be achieved by trans…
▽ More
Accurate real-time traffic forecast is critical for intelligent transportation systems (ITS) and it serves as the cornerstone of various smart mobility applications. Though this research area is dominated by deep learning, recent studies indicate that the accuracy improvement by develo** new model structures is becoming marginal. Instead, we envision that the improvement can be achieved by transferring the "forecasting-related knowledge" across cities with different data distributions and network topologies. To this end, this paper aims to propose a novel transferable traffic forecasting framework: Domain Adversarial Spatial-Temporal Network (DASTNet). DASTNet is pre-trained on multiple source networks and fine-tuned with the target network's traffic data. Specifically, we leverage the graph representation learning and adversarial domain adaptation techniques to learn the domain-invariant node embeddings, which are further incorporated to model the temporal traffic data. To the best of our knowledge, we are the first to employ adversarial multi-domain adaptation for network-wide traffic forecasting problems. DASTNet consistently outperforms all state-of-the-art baseline methods on three benchmark datasets. The trained DASTNet is applied to Hong Kong's new traffic detectors, and accurate traffic predictions can be delivered immediately (within one day) when the detector is available. Overall, this study suggests an alternative to enhance the traffic forecasting methods and provides practical implications for cities lacking historical traffic data.
△ Less
Submitted 19 August, 2022; v1 submitted 7 February, 2022;
originally announced February 2022.
-
Turning Traffic Monitoring Cameras into Intelligent Sensors for Traffic Density Estimation
Authors:
Zijian Hu,
William H. K. Lam,
S. C. Wong,
Andy H. F. Chow,
Wei Ma
Abstract:
Accurate traffic state information plays a pivotal role in the Intelligent Transportation Systems (ITS), and it is an essential input to various smart mobility applications such as signal coordination and traffic flow prediction. The current practice to obtain the traffic state information is through specialized sensors such as loop detectors and speed cameras. In most metropolitan areas, traffic…
▽ More
Accurate traffic state information plays a pivotal role in the Intelligent Transportation Systems (ITS), and it is an essential input to various smart mobility applications such as signal coordination and traffic flow prediction. The current practice to obtain the traffic state information is through specialized sensors such as loop detectors and speed cameras. In most metropolitan areas, traffic monitoring cameras have been installed to monitor the traffic conditions on arterial roads and expressways, and the collected videos or images are mainly used for visual inspection by traffic engineers. Unfortunately, the data collected from traffic monitoring cameras are affected by the 4L characteristics: Low frame rate, Low resolution, Lack of annotated data, and Located in complex road environments. Therefore, despite the great potentials of the traffic monitoring cameras, the 4L characteristics hinder them from providing useful traffic state information (e.g., speed, flow, density). This paper focuses on the traffic density estimation problem as it is widely applicable to various traffic surveillance systems. To the best of our knowledge, there is a lack of the holistic framework for addressing the 4L characteristics and extracting the traffic density information from traffic monitoring camera data. In view of this, this paper proposes a framework for estimating traffic density using uncalibrated traffic monitoring cameras with 4L characteristics. The proposed framework consists of two major components: camera calibration and vehicle detection. The camera calibration method estimates the actual length between pixels in the images and videos, and the vehicle counts are extracted from the deep-learning-based vehicle detection method. Combining the two components, high-granular traffic density can be estimated. To validate the proposed framework, two case studies were conducted in Hong Kong and Sacramento. The results show that the Mean Absolute Error (MAE) in camera calibration is less than 0.2 meters out of 6 meters, and the accuracy of vehicle detection under various conditions is approximately 90%. Overall, the MAE for the estimated density is 9.04 veh/km/lane in Hong Kong and 1.30 veh/km/lane in Sacramento. The research outcomes can be used to calibrate the speed-density fundamental diagrams, and the proposed framework can provide accurate and real-time traffic information without installing additional sensors.
△ Less
Submitted 29 October, 2021;
originally announced November 2021.
-
An Output Containment Approach to Cooperative Control of Multiple Unmanned and Manned Vehicles
Authors:
Wang Shimin,
Jiang Simin,
Zhan Zhi,
Wu Yuanqing,
William H. K. Lam,
Zhong Renxin
Abstract:
This paper investigates the cooperative control of multiple unmanned and manned vehicles via an output containment control approach for heterogeneous discrete-time multiagent systems. The unmanned vehicles act as leading vehicles to guide the manned vehicles, i.e., following vehicles. The objective is to develop a distributed output feedback control law such that the output of the following vehicl…
▽ More
This paper investigates the cooperative control of multiple unmanned and manned vehicles via an output containment control approach for heterogeneous discrete-time multiagent systems. The unmanned vehicles act as leading vehicles to guide the manned vehicles, i.e., following vehicles. The objective is to develop a distributed output feedback control law such that the output of the following vehicles can converge to the convex hull spanned by the output of the leading vehicles exponentially. The convex hull formed by the output of the leading vehicles and the system matrix of leading vehicles are estimated via a distributed containment observer. Based on this observer, a distributed dynamic output feedback control protocol is first devised for heterogeneous discrete-time multi-agent systems using only neighboring relative output information. The proof is depicted by showing certain output containment errors converge to zero exponentially, which indicates the containment control objective is well achieved. A distributed dynamic state-feedback control law is deduced as a special case of the output feedback control. Finally, numerical simulations with application to cooperative control of multiple vehicles validate the effectiveness and the computational feasibility of the proposed control protocols.
△ Less
Submitted 16 February, 2021; v1 submitted 24 May, 2020;
originally announced May 2020.
