-
Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions
Authors:
Desong Du,
Shaohang Han,
Naiming Qi,
Haitham Bou Ammar,
Jun Wang,
Wei Pan
Abstract:
Reinforcement learning (RL) exhibits impressive performance when managing complicated control tasks for robots. However, its wide application to physical robots is limited by the absence of strong safety guarantees. To overcome this challenge, this paper explores the control Lyapunov barrier function (CLBF) to analyze the safety and reachability solely based on data without explicitly employing a…
▽ More
Reinforcement learning (RL) exhibits impressive performance when managing complicated control tasks for robots. However, its wide application to physical robots is limited by the absence of strong safety guarantees. To overcome this challenge, this paper explores the control Lyapunov barrier function (CLBF) to analyze the safety and reachability solely based on data without explicitly employing a dynamic model. We also proposed the Lyapunov barrier actor-critic (LBAC), a model-free RL algorithm, to search for a controller that satisfies the data-based approximation of the safety and reachability conditions. The proposed approach is demonstrated through simulation and real-world robot control experiments, i.e., a 2D quadrotor navigation task. The experimental findings reveal this approach's effectiveness in reachability and safety, surpassing other model-free RL methods.
△ Less
Submitted 16 May, 2023;
originally announced May 2023.
-
Towards automated optimisation of residual convolutional neural networks for electrocardiogram classification
Authors:
Zeineb Fki,
Boudour Ammar,
Mounir Ben Ayed
Abstract:
The interpretation of the electrocardiogram (ECG) gives clinical information and helps in assessing heart function. There are distinct ECG patterns associated with a specific class of arrythmia. The convolutional neural network is currently one of the most commonly employed deep learning algorithms for ECG processing. However, deep learning models require many hyperparameters to tune. Selecting an…
▽ More
The interpretation of the electrocardiogram (ECG) gives clinical information and helps in assessing heart function. There are distinct ECG patterns associated with a specific class of arrythmia. The convolutional neural network is currently one of the most commonly employed deep learning algorithms for ECG processing. However, deep learning models require many hyperparameters to tune. Selecting an optimal or best hyperparameter for the convolutional neural network algorithm is a highly challenging task. Often, we end up tuning the model manually with different possible ranges of values until a best fit model is obtained. Automatic hyperparameters tuning using Bayesian optimisation (BO) and evolutionary algorithms can provide an effective solution to current labour-intensive manual configuration approaches. In this paper, we propose to optimise the Residual one Dimensional Convolutional Neural Network model (R-1D-CNN) at two levels. At the first level, a residual convolutional layer and one-dimensional convolutional neural layers are trained to learn patient-specific ECG features over which multilayer perceptron layers can learn to produce the final class vectors of each input. This level is manual and aims to lower the search space. The second level is automatic and based on our proposed BO-based algorithm. Our proposed optimised R-1D-CNN architecture is evaluated on two publicly available ECG Datasets. Comparative experimental results demonstrate that our BO-based algorithm achieves an optimal rate of 99.95%, while the baseline model achieves 99.70% for the MIT-BIH database. Moreover, experiments demonstrate that the proposed architecture fine-tuned with BO achieves a higher accuracy than the other proposed architectures. Our optimised architecture achieves excellent results compared to previous works on benchmark datasets.
△ Less
Submitted 28 January, 2022; v1 submitted 11 December, 2021;
originally announced December 2021.
-
SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
Authors:
Ming Zhou,
Jun Luo,
Julian Villella,
Yaodong Yang,
David Rusu,
Jiayu Miao,
Weinan Zhang,
Montgomery Alban,
Iman Fadakar,
Zheng Chen,
Aurora Chongxi Huang,
Ying Wen,
Kimia Hassanzadeh,
Daniel Graves,
Dong Chen,
Zhengbang Zhu,
Nhat Nguyen,
Mohamed Elsayed,
Kun Shao,
Sanjeevan Ahilan,
Baokuan Zhang,
Jiannan Wu,
Zhengang Fu,
Kasra Rezaee,
Peyman Yadmellat
, et al. (12 additional authors not shown)
Abstract:
Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Despite more than a decade of research and development, the problem of how to competently interact with diverse road users in diverse scenarios remains largely unsolved. Learning methods have much to offer towards solving this problem. But they require a realistic multi-agent simulator that generates diverse a…
▽ More
Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Despite more than a decade of research and development, the problem of how to competently interact with diverse road users in diverse scenarios remains largely unsolved. Learning methods have much to offer towards solving this problem. But they require a realistic multi-agent simulator that generates diverse and competent driving interactions. To meet this need, we develop a dedicated simulation platform called SMARTS (Scalable Multi-Agent RL Training School). SMARTS supports the training, accumulation, and use of diverse behavior models of road users. These are in turn used to create increasingly more realistic and diverse interactions that enable deeper and broader research on multi-agent interaction. In this paper, we describe the design goals of SMARTS, explain its basic architecture and its key features, and illustrate its use through concrete multi-agent experiments on interactive scenarios. We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving. Our code is available at https://github.com/huawei-noah/SMARTS.
△ Less
Submitted 31 October, 2020; v1 submitted 19 October, 2020;
originally announced October 2020.
