-
Random Network Distillation Based Deep Reinforcement Learning for AGV Path Planning
Authors:
Huilin Yin,
Shengkai Su,
Yinjia Lin,
Pengju Zhen,
Karin Festl,
Daniel Watzenig
Abstract:
With the flourishing development of intelligent warehousing systems, the technology of Automated Guided Vehicle (AGV) has experienced rapid growth. Within intelligent warehousing environments, AGV is required to safely and rapidly plan an optimal path in complex and dynamic environments. Most research has studied deep reinforcement learning to address this challenge. However, in the environments w…
▽ More
With the flourishing development of intelligent warehousing systems, the technology of Automated Guided Vehicle (AGV) has experienced rapid growth. Within intelligent warehousing environments, AGV is required to safely and rapidly plan an optimal path in complex and dynamic environments. Most research has studied deep reinforcement learning to address this challenge. However, in the environments with sparse extrinsic rewards, these algorithms often converge slowly, learn inefficiently or fail to reach the target. Random Network Distillation (RND), as an exploration enhancement, can effectively improve the performance of proximal policy optimization, especially enhancing the additional intrinsic rewards of the AGV agent which is in sparse reward environments. Moreover, most of the current research continues to use 2D grid mazes as experimental environments. These environments have insufficient complexity and limited action sets. To solve this limitation, we present simulation environments of AGV path planning with continuous actions and positions for AGVs, so that it can be close to realistic physical scenarios. Based on our experiments and comprehensive analysis of the proposed method, the results demonstrate that our proposed method enables AGV to more rapidly complete path planning tasks with continuous actions in our environments. A video of part of our experiments can be found at https://youtu.be/lwrY9YesGmw.
△ Less
Submitted 18 April, 2024;
originally announced April 2024.
-
Multi-objective path tracking control for car-like vehicles with differentially bounded n-smooth output
Authors:
Karin Festl,
Michael Stolz,
Daniel Watzenig
Abstract:
When designing path tracking controllers for car-like vehicles, two main aspects are the tracking performance and the characteristics of the actuation signal. Our work bases on an existing variable structure controller that was designed with the geometrically optimal solution of a Dubins car, but with chattering on the output. In this contribution, we extend this approach to achieve an actuation s…
▽ More
When designing path tracking controllers for car-like vehicles, two main aspects are the tracking performance and the characteristics of the actuation signal. Our work bases on an existing variable structure controller that was designed with the geometrically optimal solution of a Dubins car, but with chattering on the output. In this contribution, we extend this approach to achieve an actuation signal that is n-smooth and differentially bounded. While the global stability under matched disturbances is maintained, the finite time reaching behavior is exchanged for asymptotic convergence. With the $n$-smooth output, n new parameters are introduced, weighing the control characteristics between the tracking performance and the magnitude of the steering angles derivatives. The controller is also evaluated in simulation, demonstrating the tuning capabilities, as well as the reaching and tracking behavior. The main contribution of this work is a control law designed to produce a smooth steering angle with implicit satisfaction of bounds on its derivatives.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
-
Performance of Graph Database Management Systems as route planning solutions for different data and usage characteristics
Authors:
Karin Festl,
Patrick Promitzer,
Daniel Watzenig,
Huilin Yin
Abstract:
Graph databases have grown in popularity in recent years as they are able to efficiently store and query complex relationships between data. Incidentally, navigation data and road networks can be processed, sampled or modified efficiently when stored as a graph. As a result, graph databases are a solution for solving route planning tasks that comes more and more to the attention of developers of a…
▽ More
Graph databases have grown in popularity in recent years as they are able to efficiently store and query complex relationships between data. Incidentally, navigation data and road networks can be processed, sampled or modified efficiently when stored as a graph. As a result, graph databases are a solution for solving route planning tasks that comes more and more to the attention of developers of autonomous vehicles. To achieve a computational performance that enables the realization of route planning on large road networks or for a great number of agents concurrently, several aspects need to be considered in the design of the solution. Based on a concrete use case for centralized route planning, we discuss the characteristics and properties of a use case that can significantly influence the computational effort or efficiency of the database management system. Subsequently we evaluate the performance of both Neo4j and ArangoDB depending on these properties. With these results, it is not only possible to choose the most suitable database system but also to improve the resulting performance by addressing relevant aspects in the design of the application.
△ Less
Submitted 12 June, 2023;
originally announced June 2023.
