-
TASAC: a twin-actor reinforcement learning framework with stochastic policy for batch process control
Authors:
Tanuja Joshi,
Hariprasad Kodamana,
Harikumar Kandath,
Niket Kaisare
Abstract:
Due to their complex nonlinear dynamics and batch-to-batch variability, batch processes pose a challenge for process control. Due to the absence of accurate models and resulting plant-model mismatch, these problems become harder to address for advanced model-based control strategies. Reinforcement Learning (RL), wherein an agent learns the policy by directly interacting with the environment, offer…
▽ More
Due to their complex nonlinear dynamics and batch-to-batch variability, batch processes pose a challenge for process control. Due to the absence of accurate models and resulting plant-model mismatch, these problems become harder to address for advanced model-based control strategies. Reinforcement Learning (RL), wherein an agent learns the policy by directly interacting with the environment, offers a potential alternative in this context. RL frameworks with actor-critic architecture have recently become popular for controlling systems where state and action spaces are continuous. It has been shown that an ensemble of actor and critic networks further helps the agent learn better policies due to the enhanced exploration due to simultaneous policy learning. To this end, the current study proposes a stochastic actor-critic RL algorithm, termed Twin Actor Soft Actor-Critic (TASAC), by incorporating an ensemble of actors for learning, in a maximum entropy framework, for batch process control.
△ Less
Submitted 2 May, 2022; v1 submitted 22 April, 2022;
originally announced April 2022.
-
Robust Consensus of Higher-Order Multi-Agent Systems With Attrition and Inclusion of Agents and Switching Topologies
Authors:
**raj V Pushpangathan,
Harikumar Kandath,
Rajdeep Dutta,
Rajarshi Bardhan,
J. Senthilnath
Abstract:
Some of the issues associated with the practical applications of consensus of multi-agent systems (MAS) include switching topologies, attrition and inclusion of agents from an existing network, and model uncertainties of agents. In this paper, a single distributed dynamic state-feedback protocol referred to as the Robust Attrition-Inclusion Distributed Dynamic (RAIDD) consensus protocol, is synthe…
▽ More
Some of the issues associated with the practical applications of consensus of multi-agent systems (MAS) include switching topologies, attrition and inclusion of agents from an existing network, and model uncertainties of agents. In this paper, a single distributed dynamic state-feedback protocol referred to as the Robust Attrition-Inclusion Distributed Dynamic (RAIDD) consensus protocol, is synthesized for achieving the consensus of MAS with attrition and inclusion of linear time-invariant higher-order uncertain homogeneous agents and switching topologies. A state consensus problem termed as the Robust Attrition-Inclusion (RAI) consensus problem is formulated to find this RAIDD consensus protocol. To solve this RAI consensus problem, first, the sufficient condition for the existence of the RAIDD protocol is obtained using the $ν$-gap metric-based simultaneous stabilization approach. Next, the RAIDD consensus protocol is attained using the Glover-McFarlane robust stabilization method if the sufficient condition is satisfied. The performance of this RAIDD protocol is validated by numerical simulations.
△ Less
Submitted 13 February, 2022;
originally announced February 2022.
-
Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control
Authors:
Tanuja Joshi,
Shikhar Makker,
Hariprasad Kodamana,
Harikumar Kandath
Abstract:
Control of batch processes is a difficult task due to their complex nonlinear dynamics and unsteady-state operating conditions within batch and batch-to-batch. It is expected that some of these challenges can be addressed by develo** control strategies that directly interact with the process and learning from experiences. Recent studies in the literature have indicated the advantage of having an…
▽ More
Control of batch processes is a difficult task due to their complex nonlinear dynamics and unsteady-state operating conditions within batch and batch-to-batch. It is expected that some of these challenges can be addressed by develo** control strategies that directly interact with the process and learning from experiences. Recent studies in the literature have indicated the advantage of having an ensemble of actors in actor-critic Reinforcement Learning (RL) frameworks for improving the policy. The present study proposes an actor-critic RL algorithm, namely, twin actor twin delayed deep deterministic policy gradient (TATD3), by incorporating twin actor networks in the existing twin-delayed deep deterministic policy gradient (TD3) algorithm for the continuous control. In addition, two types of novel reward functions are also proposed for TATD3 controller. We showcase the efficacy of the TATD3 based controller for various batch process examples by comparing it with some of the existing RL algorithms presented in the literature.
△ Less
Submitted 17 September, 2021; v1 submitted 25 February, 2021;
originally announced February 2021.
-
Gap Reduced Minimum Error Robust Simultaneous Estimation For Unstable Nano Air Vehicle
Authors:
**raj V Pushpangathan,
Harikumar Kandath,
Suresh Sundaram,
Narasimhan Sundararajan
Abstract:
This paper proposes a novel Gap Reduced Minimum Error Robust Simultaneous (GRMERS) estimator for resource-constrained Nano Aerial Vehicle (NAV) that enables a single estimator to provide simultaneous and robust estimation for a given N unstable and uncertain NAV plant models. The estimated full state feedback enables a stable flight for NAV. The GRMERS estimator is implemented utilizing a Minimum…
▽ More
This paper proposes a novel Gap Reduced Minimum Error Robust Simultaneous (GRMERS) estimator for resource-constrained Nano Aerial Vehicle (NAV) that enables a single estimator to provide simultaneous and robust estimation for a given N unstable and uncertain NAV plant models. The estimated full state feedback enables a stable flight for NAV. The GRMERS estimator is implemented utilizing a Minimum Error Robust Simultaneous (MERS) estimator and Gap Reducing (GR) compensators. The MERS estimator provides robust simultaneous estimation with minimal largest worst-case estimation error even in the presence of a bounded energy exogenous disturbance signal. The GR compensators reduce the gap between the graphs of N linear plant models to decrease the estimation error generated by the MERS estimator. A sufficient condition for the existence of a simultaneous estimator is established using LMIs and robust estimation theory. Further, MERS estimator and GR compensator design are formulated as non-convex tractable optimization problems and are solved using the population-based genetic algorithms. The performance of the GRMERS estimator consisting of MERS estimator and GR compensators from the population-based genetic algorithms is validated through simulation studies. The study results indicate that a single GRMERS estimator can produce state estimates with reduced errors for all flight conditions. The results indicate that the single GRMERS estimator is robust than the individually designed H inifinity filters.
△ Less
Submitted 12 December, 2020;
originally announced December 2020.
-
Real-time UAV Complex Missions Leveraging Self-Adaptive Controller with Elastic Structure
Authors:
Mohamad Abdul Hady,
Basaran Bahadir Kocer,
Harikumar Kandath,
Mahardhika Pratama
Abstract:
The expectation of unmanned air vehicles (UAVs) pushes the operation environment to narrow spaces, where the systems may fly very close to an object and perform an interaction. This phase brings the variation in UAV dynamics: thrust and drag coefficient of the propellers might change under different proximity. At the same time, UAVs may need to operate under external disturbances to follow time-ba…
▽ More
The expectation of unmanned air vehicles (UAVs) pushes the operation environment to narrow spaces, where the systems may fly very close to an object and perform an interaction. This phase brings the variation in UAV dynamics: thrust and drag coefficient of the propellers might change under different proximity. At the same time, UAVs may need to operate under external disturbances to follow time-based trajectories. Under these challenging conditions, a standard controller approach may not handle all missions with a fixed structure, where there may be a need to adjust its parameters for each different case. With these motivations, practical implementation and evaluation of an autonomous controller applied to a quadrotor UAV are proposed in this work. A self-adaptive controller based on a composite control scheme where a combination of sliding mode control (SMC) and evolving neuro-fuzzy control is used. The parameter vector of the neuro-fuzzy controller is updated adaptively based on the sliding surface of the SMC. The autonomous controller possesses a new elastic structure, where the number of fuzzy rules keeps growing or get pruned based on bias and variance balance. The interaction of the UAV is experimentally evaluated in real time considering the ground effect, ceiling effect and flight through a strong fan-generated wind while following time-based trajectories.
△ Less
Submitted 26 April, 2020; v1 submitted 18 July, 2019;
originally announced July 2019.
-
Robust simultaneous stabilization and decoupling of unstable adversely coupled uncertain resource constraints plants of a nano air vehicle
Authors:
**raj V. Pushpangathan,
Harikumar Kandath,
Suresh Sundaram
Abstract:
The plants of nano air vehicles (NAVs) are generally unstable, adversely coupled, and uncertain. Besides, the autopilot hardware of a NAV has limited sensing and computational capabilities. Hence, these vehicles need a single controller referred to as Robust Simultaneously Stabilizing Decoupling (RSSD) output feedback controller that achieves simultaneous stabilization, desired decoupling, robustn…
▽ More
The plants of nano air vehicles (NAVs) are generally unstable, adversely coupled, and uncertain. Besides, the autopilot hardware of a NAV has limited sensing and computational capabilities. Hence, these vehicles need a single controller referred to as Robust Simultaneously Stabilizing Decoupling (RSSD) output feedback controller that achieves simultaneous stabilization, desired decoupling, robustness, and performance for a finite set of unstable multi-input-multi-output adversely coupled uncertain plants. To synthesize a RSSD output feedback controller, a new method that is based on a central plant is proposed in this paper. Given a finite set of plants for simultaneous stabilization, we considered a plant in this set that has the smallest maximum $v-$gap metric as the central plant. Following this, the sufficient condition for the existence of a simultaneous stabilizing controller associated with such a plant is described. The decoupling feature is then appended to this controller using the properties of the eigenstructure assignment method.
Afterward, the sufficient conditions for the existence of a RSSD output feedback controller are obtained. Using these sufficient conditions, a new optimization problem for the synthesis of a RSSD output feedback controller is formulated. To solve this optimization problem, a new genetic algorithm based offline iterative algorithm is developed. The effectiveness of this iterative algorithm is then demonstrated by generating a RSSD controller for a fixed-wing nano air vehicle. The performance of this controller is validated through numerical and hardware-in-the-loop simulations.
△ Less
Submitted 3 September, 2020; v1 submitted 1 May, 2019;
originally announced May 2019.
