-
Online Policies for Real-Time Control Using MRAC-RL
Authors:
Anubhav Guha,
Anuradha Annaswamy
Abstract:
In this paper, we propose the Model Reference Adaptive Control & Reinforcement Learning (MRAC-RL) approach to develo** online policies for systems in which modeling errors occur in real-time. Although reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems, discrepancies between simulated dynamics and the true target dynamics can cause…
▽ More
In this paper, we propose the Model Reference Adaptive Control & Reinforcement Learning (MRAC-RL) approach to develo** online policies for systems in which modeling errors occur in real-time. Although reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems, discrepancies between simulated dynamics and the true target dynamics can cause trained policies to fail to generalize and adapt appropriately when deployed in the real-world. The MRAC-RL framework generates online policies by utilizing an inner-loop adaptive controller together with a simulation-trained outer-loop RL policy. This structure allows MRAC-RL to adapt and operate effectively in a target environment, even when parametric uncertainties exists. We propose a set of novel MRAC algorithms, apply them to a class of nonlinear systems, derive the associated control laws, provide stability guarantees for the resulting closed-loop system, and show that the adaptive tracking objective is achieved. Using a simulation study of an automated quadrotor landing task, we demonstrate that the MRAC-RL approach improves upon state-of-the-art RL algorithms and techniques through the generation of online policies.
△ Less
Submitted 30 March, 2021;
originally announced March 2021.
-
MRAC-RL: A Framework for On-Line Policy Adaptation Under Parametric Model Uncertainty
Authors:
Anubhav Guha,
Anuradha Annaswamy
Abstract:
Reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems. For many such systems, these policies are trained in a simulated environment. Due to discrepancies between the simulated model and the true system dynamics, RL trained policies often fail to generalize and adapt appropriately when deployed in the real-world environment. Current res…
▽ More
Reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems. For many such systems, these policies are trained in a simulated environment. Due to discrepancies between the simulated model and the true system dynamics, RL trained policies often fail to generalize and adapt appropriately when deployed in the real-world environment. Current research in bridging this sim-to-real gap has largely focused on improvements in simulation design and on the development of improved and specialized RL algorithms for robust control policy generation. In this paper we apply principles from adaptive control and system identification to develop the model-reference adaptive control & reinforcement learning (MRAC-RL) framework. We propose a set of novel MRAC algorithms applicable to a broad range of linear and nonlinear systems, and derive the associated control laws. The MRAC-RL framework utilizes an inner-loop adaptive controller that allows a simulation-trained outer-loop policy to adapt and operate effectively in a test environment, even when parametric model uncertainty exists. We demonstrate that the MRAC-RL approach improves upon state-of-the-art RL algorithms in develo** control policies that can be applied to systems with modeling errors.
△ Less
Submitted 20 November, 2020;
originally announced November 2020.
-
Multimodal Noisy Segmentation based fragmented burn scars identification in Amazon Rainforest
Authors:
Satyam Mohla,
Sidharth Mohla,
Anupam Guha,
Biplab Banerjee
Abstract:
Detection of burn marks due to wildfires in inaccessible rain forests is important for various disaster management and ecological studies. The fragmented nature of arable landscapes and diverse crop** patterns often thwart the precise map** of burn scars. Recent advances in remote-sensing and availability of multimodal data offer a viable solution to this map** problem. However, the task to…
▽ More
Detection of burn marks due to wildfires in inaccessible rain forests is important for various disaster management and ecological studies. The fragmented nature of arable landscapes and diverse crop** patterns often thwart the precise map** of burn scars. Recent advances in remote-sensing and availability of multimodal data offer a viable solution to this map** problem. However, the task to segment burn marks is difficult because of its indistinguishably with similar looking land patterns, severe fragmented nature of burn marks and partially labelled noisy datasets. In this work we present AmazonNET -- a convolutional based network that allows extracting of burn patters from multimodal remote sensing images. The network consists of UNet: a well-known encoder decoder type of architecture with skip connections commonly used in biomedical segmentation. The proposed framework utilises stacked RGB-NIR channels to segment burn scars from the pastures by training on a new weakly labelled noisy dataset from Amazonia. Our model illustrates superior performance by correctly identifying partially labelled burn scars and rejecting incorrectly labelled samples, demonstrating our approach as one of the first to effectively utilise deep learning based segmentation models in multimodal burn scar identification.
△ Less
Submitted 9 September, 2020;
originally announced September 2020.
