-
Fleet Size and Spill for UAM Operation under Uncertain Demand
Authors:
Shangqing Cao,
Xuan Jiang,
Emin Burak Onat,
Bo Zou,
Mark Hansen,
Raja Sengupta,
Anjan Chakrabarty
Abstract:
Variation and imbalance in demand poses significant challenges to Urban Air Mobility (UAM) operations, affecting strategic decisions such as fleet sizing. To study the implications of demand variation on UAM fleet operations, we propose a stochastic passenger arrival time generation model that uses real-world data to infer demand distributions, and two integer programs that compute the zero-spill…
▽ More
Variation and imbalance in demand poses significant challenges to Urban Air Mobility (UAM) operations, affecting strategic decisions such as fleet sizing. To study the implications of demand variation on UAM fleet operations, we propose a stochastic passenger arrival time generation model that uses real-world data to infer demand distributions, and two integer programs that compute the zero-spill fleet size and the spill-minimizing flight schedules and charging policies, respectively. Our numerical experiment on a two-vertiport network shows that spill in relatively inelastic to fleet size and that the driving factor behind spill is the imbalance in demand.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
A Simulation-Optimization Framework for Develo** Wind-Resilient AAM Networks
Authors:
Emin Burak Onat,
Shangqing Cao,
Raiyan Rizwan,
Xuan Jiang,
Mark Hansen,
Raja Sengupta,
Anjan Chakrabarty
Abstract:
Environmental factors pose a significant challenge to the operational efficiency and safety of advanced air mobility (AAM) networks. This paper presents a simulation-optimization framework that dynamically integrates wind variability into AAM operations. We employ a nonlinear charging model within a multi-vertiport environment to optimize fleet size and scheduling. Our framework assesses the impac…
▽ More
Environmental factors pose a significant challenge to the operational efficiency and safety of advanced air mobility (AAM) networks. This paper presents a simulation-optimization framework that dynamically integrates wind variability into AAM operations. We employ a nonlinear charging model within a multi-vertiport environment to optimize fleet size and scheduling. Our framework assesses the impact of wind on operational parameters, providing strategies to enhance the resilience of AAM ecosystems. The results demonstrate that wind conditions exert significant influence on fleet size even for short-distance flights, their impact on fleet size and energy requirements becomes more pronounced over longer distances. Efficient management of fleet size and charging policies, particularly for long-distance networks, is needed to accommodate the variability of wind conditions effectively.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
MPC of Uncertain Nonlinear Systems with Meta-Learning for Fast Adaptation of Neural Predictive Models
Authors:
Jiaqi Yan,
Ankush Chakrabarty,
Alisa Rupenyan,
John Lygeros
Abstract:
In this paper, we consider the problem of reference tracking in uncertain nonlinear systems. A neural State-Space Model (NSSM) is used to approximate the nonlinear system, where a deep encoder network learns the nonlinearity from data, and a state-space component captures the temporal relationship. This transforms the nonlinear system into a linear system in a latent space, enabling the applicatio…
▽ More
In this paper, we consider the problem of reference tracking in uncertain nonlinear systems. A neural State-Space Model (NSSM) is used to approximate the nonlinear system, where a deep encoder network learns the nonlinearity from data, and a state-space component captures the temporal relationship. This transforms the nonlinear system into a linear system in a latent space, enabling the application of model predictive control (MPC) to determine effective control actions. Our objective is to design the optimal controller using limited data from the \textit{target system} (the system of interest). To this end, we employ an implicit model-agnostic meta-learning (iMAML) framework that leverages information from \textit{source systems} (systems that share similarities with the target system) to expedite training in the target system and enhance its control performance. The framework consists of two phases: the (offine) meta-training phase learns a aggregated NSSM using data from source systems, and the (online) meta-inference phase quickly adapts this aggregated model to the target system using only a few data points and few online training iterations, based on local loss function gradients. The iMAML algorithm exploits the implicit function theorem to exactly compute the gradient during training, without relying on the entire optimization path. By focusing solely on the optimal solution, rather than the path, we can meta-train with less storage complexity and fewer approximations than other contemporary meta-learning algorithms. We demonstrate through numerical examples that our proposed method can yield accurate predictive models by adaptation, resulting in a downstream MPC that outperforms several baselines.
△ Less
Submitted 18 April, 2024;
originally announced April 2024.
-
Fortifying Fully Convolutional Generative Adversarial Networks for Image Super-Resolution Using Divergence Measures
Authors:
Arkaprabha Basu,
Kushal Bose,
Sankha Subhra Mullick,
Anish Chakrabarty,
Swagatam Das
Abstract:
Super-Resolution (SR) is a time-hallowed image processing problem that aims to improve the quality of a Low-Resolution (LR) sample up to the standard of its High-Resolution (HR) counterpart. We aim to address this by introducing Super-Resolution Generator (SuRGe), a fully-convolutional Generative Adversarial Network (GAN)-based architecture for SR. We show that distinct convolutional features obta…
▽ More
Super-Resolution (SR) is a time-hallowed image processing problem that aims to improve the quality of a Low-Resolution (LR) sample up to the standard of its High-Resolution (HR) counterpart. We aim to address this by introducing Super-Resolution Generator (SuRGe), a fully-convolutional Generative Adversarial Network (GAN)-based architecture for SR. We show that distinct convolutional features obtained at increasing depths of a GAN generator can be optimally combined by a set of learnable convex weights to improve the quality of generated SR samples. In the process, we employ the Jensen-Shannon and the Gromov-Wasserstein losses respectively between the SR-HR and LR-SR pairs of distributions to further aid the generator of SuRGe to better exploit the available information in an attempt to improve SR. Moreover, we train the discriminator of SuRGe with the Wasserstein loss with gradient penalty, to primarily prevent mode collapse. The proposed SuRGe, as an end-to-end GAN workflow tailor-made for super-resolution, offers improved performance while maintaining low inference time. The efficacy of SuRGe is substantiated by its superior performance compared to 18 state-of-the-art contenders on 10 benchmark datasets.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
Evaluating eVTOL Network Performance and Fleet Dynamics through Simulation-Based Analysis
Authors:
Emin Burak Onat,
Vishwanath Bulusu,
Anjan Chakrabarty,
Mark Hansen,
Raja Sengupta,
Banavar Sridar
Abstract:
Urban Air Mobility (UAM) represents a promising solution for future transportation. In this study, we introduce VertiSim, an advanced event-driven simulator developed to evaluate e-VTOL transportation networks. Uniquely, VertiSim simultaneously models passenger, aircraft, and energy flows, reflecting the interrelated complexities of UAM systems. We utilized VertiSim to assess 19 operational scenar…
▽ More
Urban Air Mobility (UAM) represents a promising solution for future transportation. In this study, we introduce VertiSim, an advanced event-driven simulator developed to evaluate e-VTOL transportation networks. Uniquely, VertiSim simultaneously models passenger, aircraft, and energy flows, reflecting the interrelated complexities of UAM systems. We utilized VertiSim to assess 19 operational scenarios serving a daily demand for 2,834 passengers with varying fleet sizes and vertiport distances. The study aims to support stakeholders in making informed decisions about fleet size, network design, and infrastructure development by understanding tradeoffs in passenger delay time, operational costs, and fleet utilization. Our simulations, guided by a heuristic dispatch and charge policy, indicate that fleet size significantly influences passenger delay and energy consumption within UAM networks. We find that increasing the fleet size can reduce average passenger delays, but this comes at the cost of higher operational expenses due to an increase in the number of repositioning flights. Additionally, our analysis highlights how vertiport distances impact fleet utilization: longer distances result in reduced total idle time and increased cruise and charge times, leading to more efficient fleet utilization but also longer passenger delays. These findings are important for UAM network planning, especially in balancing fleet size with vertiport capacity and operational costs. Simulator demo is available at: https://tinyurl.com/vertisim-vis
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
Safe multi-agent motion planning under uncertainty for drones using filtered reinforcement learning
Authors:
Sleiman Safaoui,
Abraham P. Vinod,
Ankush Chakrabarty,
Rien Quirynen,
Nobuyuki Yoshikawa,
Stefano Di Cairano
Abstract:
We consider the problem of safe multi-agent motion planning for drones in uncertain, cluttered workspaces. For this problem, we present a tractable motion planner that builds upon the strengths of reinforcement learning and constrained-control-based trajectory planning. First, we use single-agent reinforcement learning to learn motion plans from data that reach the target but may not be collision-…
▽ More
We consider the problem of safe multi-agent motion planning for drones in uncertain, cluttered workspaces. For this problem, we present a tractable motion planner that builds upon the strengths of reinforcement learning and constrained-control-based trajectory planning. First, we use single-agent reinforcement learning to learn motion plans from data that reach the target but may not be collision-free. Next, we use a convex optimization, chance constraints, and set-based methods for constrained control to ensure safety, despite the uncertainty in the workspace, agent motion, and sensing. The proposed approach can handle state and control constraints on the agents, and enforce collision avoidance among themselves and with static obstacles in the workspace with high probability. The proposed approach yields a safe, real-time implementable, multi-agent motion planner that is simpler to train than methods based solely on learning. Numerical simulations and experiments show the efficacy of the approach.
△ Less
Submitted 31 October, 2023;
originally announced November 2023.
-
Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems
Authors:
Truong X. Nghiem,
Ján Drgoňa,
Colin Jones,
Zoltan Nagy,
Roland Schwan,
Biswadip Dey,
Ankush Chakrabarty,
Stefano Di Cairano,
Joel A. Paulson,
Andrea Carron,
Melanie N. Zeilinger,
Wenceslao Shaw Cortez,
Draguna L. Vrabie
Abstract:
Physics-informed machine learning (PIML) is a set of methods and tools that systematically integrate machine learning (ML) algorithms with physical constraints and abstract mathematical models developed in scientific and engineering domains. As opposed to purely data-driven methods, PIML models can be trained from additional information obtained by enforcing physical laws such as energy and mass c…
▽ More
Physics-informed machine learning (PIML) is a set of methods and tools that systematically integrate machine learning (ML) algorithms with physical constraints and abstract mathematical models developed in scientific and engineering domains. As opposed to purely data-driven methods, PIML models can be trained from additional information obtained by enforcing physical laws such as energy and mass conservation. More broadly, PIML models can include abstract properties and conditions such as stability, convexity, or invariance. The basic premise of PIML is that the integration of ML and physics can yield more effective, physically consistent, and data-efficient models. This paper aims to provide a tutorial-like overview of the recent advances in PIML for dynamical system modeling and control. Specifically, the paper covers an overview of the theory, fundamental concepts and methods, tools, and applications on topics of: 1) physics-informed learning for system identification; 2) physics-informed learning for control; 3) analysis and verification of PIML models; and 4) physics-informed digital twins. The paper is concluded with a perspective on open challenges and future research opportunities.
△ Less
Submitted 24 June, 2023;
originally announced June 2023.
-
Meta-Learning of Neural State-Space Models Using Data From Similar Systems
Authors:
Ankush Chakrabarty,
Gordon Wichern,
Christopher R. Laughman
Abstract:
Deep neural state-space models (SSMs) provide a powerful tool for modeling dynamical systems solely using operational data. Typically, neural SSMs are trained using data collected from the actual system under consideration, despite the likely existence of operational data from similar systems which have previously been deployed in the field. In this paper, we propose the use of model-agnostic meta…
▽ More
Deep neural state-space models (SSMs) provide a powerful tool for modeling dynamical systems solely using operational data. Typically, neural SSMs are trained using data collected from the actual system under consideration, despite the likely existence of operational data from similar systems which have previously been deployed in the field. In this paper, we propose the use of model-agnostic meta-learning (MAML) for constructing deep encoder network-based SSMs, by leveraging a combination of archived data from similar systems (used to meta-train offline) and limited data from the actual system (used for rapid online adaptation). We demonstrate using a numerical example that meta-learning can result in more accurate neural SSM models than supervised- or transfer-learning, despite few adaptation steps and limited online data. Additionally, we show that by carefully partitioning and adapting the encoder layers while fixing the state-transition operator, we can achieve comparable performance to MAML while reducing online adaptation complexity.
△ Less
Submitted 14 November, 2022;
originally announced November 2022.
-
Optimizing Closed-Loop Performance with Data from Similar Systems: A Bayesian Meta-Learning Approach
Authors:
Ankush Chakrabarty
Abstract:
Bayesian optimization (BO) has demonstrated potential for optimizing control performance in data-limited settings, especially for systems with unknown dynamics or unmodeled performance objectives. The BO algorithm efficiently trades-off exploration and exploitation by leveraging uncertainty estimates using surrogate models. These surrogates are usually learned using data collected from the target…
▽ More
Bayesian optimization (BO) has demonstrated potential for optimizing control performance in data-limited settings, especially for systems with unknown dynamics or unmodeled performance objectives. The BO algorithm efficiently trades-off exploration and exploitation by leveraging uncertainty estimates using surrogate models. These surrogates are usually learned using data collected from the target dynamical system to be optimized. Intuitively, the convergence rate of BO is better for surrogate models that can accurately predict the target system performance. In classical BO, initial surrogate models are constructed using very limited data points, and therefore rarely yield accurate predictions of system performance. In this paper, we propose the use of meta-learning to generate an initial surrogate model based on data collected from performance optimization tasks performed on a variety of systems that are different to the target system. To this end, we employ deep kernel networks (DKNs) which are simple to train and which comprise encoded Gaussian process models that integrate seamlessly with classical BO. The effectiveness of our proposed DKN-BO approach for speeding up control system performance optimization is demonstrated using a well-studied nonlinear system with unknown dynamics and an unmodeled performance function.
△ Less
Submitted 31 October, 2022;
originally announced November 2022.
-
Data-Driven Identification of Dynamic Quality Models in Drinking Water Networks
Authors:
Shen Wang,
Ankush Chakrabarty,
Ahmad F. Taha
Abstract:
Traditional control and monitoring of water quality in drinking water distribution networks (WDN) rely on mostly model- or toolbox-driven approaches, where the network topology and parameters are assumed to be known. In contrast, system identification (SysID) algorithms for generic dynamic system models seek to approximate such models using only input-output data without relying on network paramet…
▽ More
Traditional control and monitoring of water quality in drinking water distribution networks (WDN) rely on mostly model- or toolbox-driven approaches, where the network topology and parameters are assumed to be known. In contrast, system identification (SysID) algorithms for generic dynamic system models seek to approximate such models using only input-output data without relying on network parameters. The objective of this paper is to investigate SysID algorithms for water quality model approximation. This research problem is challenging due to (i) complex water quality and reaction dynamics and (ii) the mismatch between the requirements of SysID algorithms and the properties of water quality dynamics. In this paper, we present the first attempt to identify water quality models in WDNs using only input-output experimental data and classical SysID methods without knowing any WDN parameters. Properties of water quality models are introduced, the ensuing challenges caused by these properties when identifying water quality models are discussed, and remedial solutions are given. Through case studies, we demonstrate the applicability of SysID algorithms, show the corresponding performance in terms of accuracy and computational time, and explore the possible factors impacting water quality model identification.
△ Less
Submitted 23 January, 2023; v1 submitted 13 July, 2022;
originally announced July 2022.
-
VABO: Violation-Aware Bayesian Optimization for Closed-Loop Control Performance Optimization with Unmodeled Constraints
Authors:
Wenjie Xu,
Colin N Jones,
Bratislav Svetozarevic,
Christopher R. Laughman,
Ankush Chakrabarty
Abstract:
We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. Bayesian optimization (BO) has been demonstrated effective for improving closed-loop performance by automatically tuning controller gains or reference setpoints in a model-free manner. However, BO methods have rarely been tested on dynamical systems with unmodeled constraints. In this paper, we…
▽ More
We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. Bayesian optimization (BO) has been demonstrated effective for improving closed-loop performance by automatically tuning controller gains or reference setpoints in a model-free manner. However, BO methods have rarely been tested on dynamical systems with unmodeled constraints. In this paper, we propose a violation-aware BO algorithm (VABO) that optimizes closed-loop performance while simultaneously learning constraint-feasible solutions. Unlike classical constrained BO methods which allow an unlimited constraint violations, or safe BO algorithms that are conservative and try to operate with near-zero violations, we allow budgeted constraint violations to improve constraint learning and accelerate optimization. We demonstrate the effectiveness of our proposed VABO method for energy minimization of industrial vapor compression systems.
△ Less
Submitted 14 October, 2021;
originally announced October 2021.
-
Extremum Seeking Control with an Adaptive Gain Based On Gradient Estimation Error
Authors:
Claus Danielson,
Scott A. Bortoff,
Ankush Chakrabarty
Abstract:
This paper presents an extremum seeking control algorithm with an adaptive step-size that adjusts the aggressiveness of the controller based on the quality of the gradient estimate. The adaptive step-size ensures that the integral-action produced by the gradient descent does not destabilize the closed-loop system. To quantify the quality of the gradient estimate, we present a batch least squares e…
▽ More
This paper presents an extremum seeking control algorithm with an adaptive step-size that adjusts the aggressiveness of the controller based on the quality of the gradient estimate. The adaptive step-size ensures that the integral-action produced by the gradient descent does not destabilize the closed-loop system. To quantify the quality of the gradient estimate, we present a batch least squares estimator with a novel weighting and show that it produces bounded estimation errors, where the uncertainty is due to the curvature of the unknown cost function. The adaptive step-size then maximizes the decrease of the combined plant and controller Lyapunov function for the worst-case estimation error. We prove that our ESC is input-to-state stable with respect to the dither signal. Finally, we demonstrate our proposed ESC through five numerical examples; one illustrative, one practical, and three benchmarks.
△ Less
Submitted 18 December, 2021; v1 submitted 2 July, 2021;
originally announced July 2021.
-
Model Order Reduction for Water Quality Dynamics
Authors:
Shen Wang,
Ahmad F. Taha,
Ankush Chakrabarty,
Lina Sela,
Ahmed Abokifa
Abstract:
A state-space representation of water quality dynamics describing disinfectant (e.g., chlorine) transport dynamics in drinking water distribution networks has been recently proposed. Such representation is a byproduct of space- and time-discretization of the PDE modeling transport dynamics. This results in a large state-space dimension even for small networks with tens of nodes. Although such a st…
▽ More
A state-space representation of water quality dynamics describing disinfectant (e.g., chlorine) transport dynamics in drinking water distribution networks has been recently proposed. Such representation is a byproduct of space- and time-discretization of the PDE modeling transport dynamics. This results in a large state-space dimension even for small networks with tens of nodes. Although such a state-space model provides a model-driven approach to predict water quality dynamics, incorporating it into model-based control algorithms or state estimators for large networks is challenging and at times intractable. To that end, this paper investigates model order reduction (MOR) methods for water quality dynamics with the objective of performing post-reduction feedback control. The presented investigation focuses on reducing state-dimension by orders of magnitude, the stability of the MOR methods, and the application of these methods to model predictive control.
△ Less
Submitted 18 February, 2022; v1 submitted 21 February, 2021;
originally announced February 2021.
-
Modeling of Vertical Dipole Above Lossy Dielectric Half-Space: Characteristic Mode Theory
Authors:
Sandip Ghosal,
Arijit De,
Raed M. Shubair,
Ajay Chakrabarty
Abstract:
This work introduces a theoretical extension of the characteristic mode formulation for analysing the vertical electric dipole lying above a lossy dielectric half-space. As the conventional characteristic formulation fails to maintain the orthogonality of the characteristic field modes over the infinite sphere, an alternate modal formulation is proposed here to maintain the orthogonality for both…
▽ More
This work introduces a theoretical extension of the characteristic mode formulation for analysing the vertical electric dipole lying above a lossy dielectric half-space. As the conventional characteristic formulation fails to maintain the orthogonality of the characteristic field modes over the infinite sphere, an alternate modal formulation is proposed here to maintain the orthogonality for both the current and field modes. The modal results are found to match closely with its method of moment counterparts. Later, the modes of an isolated dipole with no ground plane have been used to predict the role of the lossy ground plane through a theory of the linear combination of the eigenvectors. The proposed formulations have been studied with different heights from the ground plane and are compared with the direct modal solutions to validate its accuracy. It helps to provide a thorough understanding of how the isolated modes interact among each other to constitute the perturbed modes in the presence of the lossy half-space. It can find application to include the lossy earth effect in the study of the lightning fields and the path loss modelling of the antennas over the lossy ground.
△ Less
Submitted 8 September, 2020;
originally announced September 2020.
-
Safe Learning-based Observers for Unknown Nonlinear Systems using Bayesian Optimization
Authors:
Ankush Chakrabarty,
Mouhacine Benosman
Abstract:
Data generated from dynamical systems with unknown dynamics enable the learning of state observers that are: robust to modeling error, computationally tractable to design, and capable of operating with guaranteed performance. In this paper, a modular design methodology is formulated, that consists of three design phases: (i) an initial robust observer design that enables one to learn the dynamics…
▽ More
Data generated from dynamical systems with unknown dynamics enable the learning of state observers that are: robust to modeling error, computationally tractable to design, and capable of operating with guaranteed performance. In this paper, a modular design methodology is formulated, that consists of three design phases: (i) an initial robust observer design that enables one to learn the dynamics without allowing the state estimation error to diverge (hence, safe); (ii) a learning phase wherein the unmodeled components are estimated using Bayesian optimization and Gaussian processes; and, (iii) a re-design phase that leverages the learned dynamics to improve convergence rate of the state estimation error. The potential of our proposed learning-based observer is demonstrated on a benchmark nonlinear system. Additionally, certificates of guaranteed estimation performance are provided.
△ Less
Submitted 25 June, 2021; v1 submitted 12 May, 2020;
originally announced May 2020.
-
Near-Field Radiation Exposure Control in Slot-Loaded Microstrip Antenna: A Characteristic Mode Approach
Authors:
Sandip Ghosal,
Arijit De,
Raed M. Shubair,
Ajay Chakrabarty
Abstract:
Microstip antenna topology is commonly loaded with a narrow slot to manipulate the resonance frequency or impedance bandwidth. However, the tuning of the resonance frequency or impedance bandwidth results in the variation of the current and field distributions. In this regard, this work adopts the concept of characteristic modes to gain an initial understanding of the perturbation mechanism of the…
▽ More
Microstip antenna topology is commonly loaded with a narrow slot to manipulate the resonance frequency or impedance bandwidth. However, the tuning of the resonance frequency or impedance bandwidth results in the variation of the current and field distributions. In this regard, this work adopts the concept of characteristic modes to gain an initial understanding of the perturbation mechanism of the rectangular patch when loaded with a slot. The performance of microstrip antennas with finite ground plane is then studied using full-wave simulation. It has been found that the distribution of the induced current density is highly dependent on the orientation of the slot The incorporation of a narrow slot suppresses the nearby orthogonal eigen mode and, as a consequence, the radiation behavior is affected. Specifically, in the presence of biological tissues in the near-field region, both antenna input impedance properties and the realized gain are dependent on the slot orientation. Different examples are included for understanding the impact of slot loading on the energy absorption by biological tissues, by calculating the the specific absorption rate (SAR). The proposed analysis facilitates the design of miniaturized antenna geometries for biomedical applications via systematic loading of narrow slots.
△ Less
Submitted 28 July, 2019;
originally announced July 2019.
-
Safe Approximate Dynamic Programming Via Kernelized Lipschitz Estimation
Authors:
Ankush Chakrabarty,
Devesh K. Jha,
Gregery T. Buzzard,
Yebin Wang,
Kyriakos Vamvoudakis
Abstract:
We develop a method for obtaining safe initial policies for reinforcement learning via approximate dynamic programming (ADP) techniques for uncertain systems evolving with discrete-time dynamics. We employ kernelized Lipschitz estimation and semidefinite programming for computing admissible initial control policies with provably high probability. Such admissible controllers enable safe initializat…
▽ More
We develop a method for obtaining safe initial policies for reinforcement learning via approximate dynamic programming (ADP) techniques for uncertain systems evolving with discrete-time dynamics. We employ kernelized Lipschitz estimation and semidefinite programming for computing admissible initial control policies with provably high probability. Such admissible controllers enable safe initialization and constraint enforcement while providing exponential stability of the equilibrium of the closed-loop system.
△ Less
Submitted 3 July, 2019;
originally announced July 2019.
-
Approximate Dynamic Programming For Linear Systems with State and Input Constraints
Authors:
Ankush Chakrabarty,
Rien Quirynen,
Claus Danielson,
Weinan Gao
Abstract:
Enforcing state and input constraints during reinforcement learning (RL) in continuous state spaces is an open but crucial problem which remains a roadblock to using RL in safety-critical applications. This paper leverages invariant sets to update control policies within an approximate dynamic programming (ADP) framework that guarantees constraint satisfaction for all time and converges to the opt…
▽ More
Enforcing state and input constraints during reinforcement learning (RL) in continuous state spaces is an open but crucial problem which remains a roadblock to using RL in safety-critical applications. This paper leverages invariant sets to update control policies within an approximate dynamic programming (ADP) framework that guarantees constraint satisfaction for all time and converges to the optimal policy (in a linear quadratic regulator sense) asymptotically. An algorithm for implementing the proposed constrained ADP approach in a data-driven manner is provided. The potential of this formalism is demonstrated via numerical examples.
△ Less
Submitted 26 June, 2019;
originally announced June 2019.
-
L2 Observers for a Class of Nonlinear Systems with Unknown Inputs
Authors:
Martin Corless,
Ankush Chakrabarty
Abstract:
We consider the problem of estimating the state and unknown input for a large class of nonlinear systems subject to unknown exogenous inputs. The exogenous inputs themselves are modeled as being generated by a nonlinear system subject to unknown inputs. The nonlinearities considered in this work are characterized by multiplier matrices that include many commonly encountered nonlinearities. We obta…
▽ More
We consider the problem of estimating the state and unknown input for a large class of nonlinear systems subject to unknown exogenous inputs. The exogenous inputs themselves are modeled as being generated by a nonlinear system subject to unknown inputs. The nonlinearities considered in this work are characterized by multiplier matrices that include many commonly encountered nonlinearities. We obtain a linear matrix inequality (LMI), that, if feasible, provides the gains for an observer which results in certified L2 performance of the error dynamics associated with the observer. We also present conditions which guarantee that the L2 norm of the error can be made arbitrarily small and investigate conditions for feasibility of the proposed LMIs.
△ Less
Submitted 21 February, 2019;
originally announced February 2019.
-
Simultaneous state and exogenous input estimation for nonlinear systems using boundary-layer sliding mode observers
Authors:
Ankush Chakrabarty,
Gregery T. Buzzard,
Stanislaw H. Zak,
Fanglai Zhu,
Ann E. Rundell
Abstract:
While sliding mode observers (SMOs) using discontinuous relays are widely analyzed, most SMOs are implemented computationally using a continuous approximation of the discontinuous relays. This approximation results in the formation of a boundary layer in a neighborhood of the sliding manifold in the observer error space. Therefore, it becomes necessary to develop methods for attenuating the effect…
▽ More
While sliding mode observers (SMOs) using discontinuous relays are widely analyzed, most SMOs are implemented computationally using a continuous approximation of the discontinuous relays. This approximation results in the formation of a boundary layer in a neighborhood of the sliding manifold in the observer error space. Therefore, it becomes necessary to develop methods for attenuating the effect of the boundary layer and guaranteeing performance bounds on the resulting state estimation error. In this paper, a method is proposed for constructing boundary-layer SMOs (BL-SMOs) with prescribed state estimation error bounds. The BL-SMO formulation is then extended to simultaneously estimate exogenous inputs (disturbance signals in the state and output vector fields), along with the system state. Two numerical examples are presented to illustrate the effectiveness of the proposed approach.
△ Less
Submitted 17 December, 2015; v1 submitted 14 July, 2015;
originally announced July 2015.
