Safe Model-based Off-policy Reinforcement Learning for Eco-Driving in Connected and Automated Hybrid Electric Vehicles
Authors:
Zhaoxuan Zhu,
Nicola Pivaro,
Shobhit Gupta,
Abhishek Gupta,
Marcello Canova
Abstract:
Connected and Automated Hybrid Electric Vehicles have the potential to reduce fuel consumption and travel time in real-world driving conditions. The eco-driving problem seeks to design optimal speed and power usage profiles based upon look-ahead information from connectivity and advanced map** features. Recently, Deep Reinforcement Learning (DRL) has been applied to the eco-driving problem. Whil…
▽ More
Connected and Automated Hybrid Electric Vehicles have the potential to reduce fuel consumption and travel time in real-world driving conditions. The eco-driving problem seeks to design optimal speed and power usage profiles based upon look-ahead information from connectivity and advanced map** features. Recently, Deep Reinforcement Learning (DRL) has been applied to the eco-driving problem. While the previous studies synthesize simulators and model-free DRL to reduce online computation, this work proposes a Safe Off-policy Model-Based Reinforcement Learning algorithm for the eco-driving problem. The advantages over the existing literature are three-fold. First, the combination of off-policy learning and the use of a physics-based model improves the sample efficiency. Second, the training does not require any extrinsic rewarding mechanism for constraint satisfaction. Third, the feasibility of trajectory is guaranteed by using a safe set approximated by deep generative models.
The performance of the proposed method is benchmarked against a baseline controller representing human drivers, a previously designed model-free DRL strategy, and the wait-and-see optimal solution. In simulation, the proposed algorithm leads to a policy with a higher average speed and a better fuel economy compared to the model-free agent. Compared to the baseline controller, the learned strategy reduces the fuel consumption by more than 21\% while kee** the average speed comparable.
△ Less
Submitted 30 January, 2022; v1 submitted 24 May, 2021;
originally announced May 2021.
A GPU Implementation of a Look-Ahead Optimal Controller for Eco-Driving Based on Dynamic Programming
Authors:
Zhaoxuan Zhu,
Shobhit Gupta,
Nicola Pivaro,
Shreshta Rajakumar Deshpande,
Marcello Canova
Abstract:
Predictive energy management of Connected and Automated Vehicles (CAVs), in particular those with multiple power sources, has the potential to significantly improve energy savings in real-world driving conditions. In particular, the eco-driving problem seeks to design optimal speed and power usage profiles based upon available information from connectivity and advanced map** features to minimize…
▽ More
Predictive energy management of Connected and Automated Vehicles (CAVs), in particular those with multiple power sources, has the potential to significantly improve energy savings in real-world driving conditions. In particular, the eco-driving problem seeks to design optimal speed and power usage profiles based upon available information from connectivity and advanced map** features to minimize the fuel consumption between two designated locations.
In this work, the eco-driving problem is formulated as a three-state receding horizon optimal control problem and solved via Dynamic Programming (DP). The optimal solution, in terms of vehicle speed and battery State of Charge (SoC) trajectories, allows a connected and automated hybrid electric vehicle to intelligently pass the signalized intersections and minimize fuel consumption over a prescribed route. To enable real-time implementation, a parallel architecture of DP is proposed for an NVIDIA GPU with CUDA programming. Simulation results indicate that the proposed optimal controller delivers more than 15% fuel economy benefits compared to a baseline control strategy and that the solver time can be reduced by more than 90% by the parallel implementation when compared to a serial implementation.
△ Less
Submitted 2 April, 2021;
originally announced April 2021.
