-
On the stability of second order gradient descent for time varying convex functions
Authors:
Travis E. Gibson,
Sawal Acharya,
Anjali Parashar,
Joseph E. Gaudio,
Anurdha M. Annaswamy
Abstract:
Gradient based optimization algorithms deployed in Machine Learning (ML) applications are often analyzed and compared by their convergence rates or regret bounds. While these rates and bounds convey valuable information they don't always directly translate to stability guarantees. Stability and similar concepts, like robustness, will become ever more important as we move towards deploying models i…
▽ More
Gradient based optimization algorithms deployed in Machine Learning (ML) applications are often analyzed and compared by their convergence rates or regret bounds. While these rates and bounds convey valuable information they don't always directly translate to stability guarantees. Stability and similar concepts, like robustness, will become ever more important as we move towards deploying models in real-time and safety critical systems. In this work we build upon the results in Gaudio et al. 2021 and Moreu and Annaswamy 2022 for second order gradient descent when applied to explicitly time varying cost functions and provide more general stability guarantees. These more general results can aid in the design and certification of these optimization schemes so as to help ensure safe and reliable deployment for real-time learning applications. We also hope that the techniques provided here will stimulate and cross-fertilize the analysis that occurs on the same algorithms from the online learning and stochastic optimization communities.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Online Algorithms and Policies Using Adaptive and Machine Learning Approaches
Authors:
Anuradha M. Annaswamy,
Anubhav Guha,
Yingnan Cui,
Sunbochen Tang,
Peter A. Fisher,
Joseph E. Gaudio
Abstract:
This paper considers the problem of real-time control and learning in dynamic systems subjected to parametric uncertainties. We propose a combination of a Reinforcement Learning (RL) based policy in the outer loop suitably chosen to ensure stability and optimality for the nominal dynamics, together with Adaptive Control (AC) in the inner loop so that in real-time AC contracts the closed-loop dynam…
▽ More
This paper considers the problem of real-time control and learning in dynamic systems subjected to parametric uncertainties. We propose a combination of a Reinforcement Learning (RL) based policy in the outer loop suitably chosen to ensure stability and optimality for the nominal dynamics, together with Adaptive Control (AC) in the inner loop so that in real-time AC contracts the closed-loop dynamics towards a stable trajectory traced out by RL. Two classes of nonlinear dynamic systems are considered, both of which are control-affine. The first class of dynamic systems utilizes equilibrium points %with expansion forms around these points and a Lyapunov approach while second class of nonlinear systems uses contraction theory. AC-RL controllers are proposed for both classes of systems and shown to lead to online policies that guarantee stability using a high-order tuner and accommodate parametric uncertainties and magnitude limits on the input. In addition to establishing a stability guarantee with real-time control, the AC-RL controller is also shown to lead to parameter learning with persistent excitation for the first class of systems. Numerical validations of all algorithms are carried out using a quadrotor landing task on a moving platform.
△ Less
Submitted 9 June, 2023; v1 submitted 13 May, 2021;
originally announced May 2021.
-
New Algorithms for Discrete-Time Parameter Estimation
Authors:
Yingnan Cui,
Joseph E. Gaudio,
Anuradha M. Annaswamy
Abstract:
We propose two algorithms for discrete-time parameter estimation, one for time-varying parameters under persistent excitation (PE) condition, another for constant parameters under no PE condition. For the first algorithm, we show that in the presence of time-varying unknown parameters, the parameter estimation error converges uniformly to a compact set under conditions of persistent excitation, wi…
▽ More
We propose two algorithms for discrete-time parameter estimation, one for time-varying parameters under persistent excitation (PE) condition, another for constant parameters under no PE condition. For the first algorithm, we show that in the presence of time-varying unknown parameters, the parameter estimation error converges uniformly to a compact set under conditions of persistent excitation, with the size of the compact set proportional to the time-variation of unknown parameters. Leveraging a projection operator, the second algorithm is shown to result in boundedness guarantees when the plant has constant unknown parameters. Simulations show better convergence results compared to recursive least squares (RLS) and comparable results to RLS with forgetting factor.
△ Less
Submitted 14 March, 2022; v1 submitted 30 March, 2021;
originally announced March 2021.
-
A High-order Tuner for Accelerated Learning and Control
Authors:
Spencer McDonald,
Yingnan Cui,
Joseph E. Gaudio,
Anuradha M. Annaswamy
Abstract:
Gradient-descent based iterative algorithms pervade a variety of problems in estimation, prediction, learning, control, and optimization. Recently iterative algorithms based on higher-order information have been explored in an attempt to lead to accelerated learning. In this paper, we explore a specific a high-order tuner that has been shown to result in stability with time-varying regressors in l…
▽ More
Gradient-descent based iterative algorithms pervade a variety of problems in estimation, prediction, learning, control, and optimization. Recently iterative algorithms based on higher-order information have been explored in an attempt to lead to accelerated learning. In this paper, we explore a specific a high-order tuner that has been shown to result in stability with time-varying regressors in linearly parametrized systems, and accelerated convergence with constant regressors. We show that this tuner continues to provide bounded parameter estimates even if the gradients are corrupted by noise. Additionally, we also show that the parameter estimates converge exponentially to a compact set whose size is dependent on noise statistics. As the HT algorithms can be applied to a wide range of problems in estimation, filtering, control, and machine learning, the result obtained in this paper represents an important extension to the topic of real-time and fast decision making.
△ Less
Submitted 23 March, 2021;
originally announced March 2021.
-
Accurate Parameter Estimation for Risk-aware Autonomous Systems
Authors:
Arnab Sarker,
Peter Fisher,
Joseph E. Gaudio,
Anuradha M. Annaswamy
Abstract:
Analysis and synthesis of safety-critical autonomous systems are carried out using models which are often dynamic. Two central features of these dynamic systems are parameters and unmodeled dynamics. This paper addresses the use of a spectral lines-based approach for estimating parameters of the dynamic model of an autonomous system. Existing literature has treated all unmodeled components of the…
▽ More
Analysis and synthesis of safety-critical autonomous systems are carried out using models which are often dynamic. Two central features of these dynamic systems are parameters and unmodeled dynamics. This paper addresses the use of a spectral lines-based approach for estimating parameters of the dynamic model of an autonomous system. Existing literature has treated all unmodeled components of the dynamic system as sub-Gaussian noise and proposed parameter estimation using Gaussian noise-based exogenous signals. In contrast, we allow the unmodeled part to have deterministic unmodeled dynamics, which are almost always present in physical systems, in addition to sub-Gaussian noise. In addition, we propose a deterministic construction of the exogenous signal in order to carry out parameter estimation. We introduce a new tool kit which employs the theory of spectral lines, retains the stochastic setting, and leads to non-asymptotic bounds on the parameter estimation error. Unlike the existing stochastic approach, these bounds are tunable through an optimal choice of the spectrum of the exogenous signal leading to accurate parameter estimation. We also show that this estimation is robust to unmodeled dynamics, a property that is not assured by the existing approach. Finally, we show that under ideal conditions with no unmodeled dynamics, the proposed approach can ensure a $\tilde{O}(\sqrt{T})$ regret, matching existing literature. Experiments are provided to support all theoretical derivations, which show that the spectral lines-based approach outperforms the Gaussian noise-based method when unmodeled dynamics are present, in terms of both parameter estimation error and Regret obtained using the parameter estimates with a Linear Quadratic Regulator in feedback.
△ Less
Submitted 16 March, 2022; v1 submitted 22 June, 2020;
originally announced June 2020.
-
Accelerated Learning with Robustness to Adversarial Regressors
Authors:
Joseph E. Gaudio,
Anuradha M. Annaswamy,
José M. Moreu,
Michael A. Bolender,
Travis E. Gibson
Abstract:
High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with accelerated learning guarantees. Such methods however, have only considered the case of static regressors. There is a significant need for parameter update algorithms…
▽ More
High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with accelerated learning guarantees. Such methods however, have only considered the case of static regressors. There is a significant need for parameter update algorithms which can be proven stable in the presence of adversarial time-varying regressors, as is commonplace in control theory. In this paper, we propose a new discrete time algorithm which 1) provides stability and asymptotic convergence guarantees in the presence of adversarial regressors by leveraging insights from adaptive control theory and 2) provides non-asymptotic accelerated learning guarantees leveraging insights from convex optimization. In particular, our algorithm reaches an $ε$ sub-optimal point in at most $\tilde{\mathcal{O}}(1/\sqrtε)$ iterations when regressors are constant - matching lower bounds due to Nesterov of $Ω(1/\sqrtε)$, up to a $\log(1/ε)$ factor and provides guaranteed bounds for stability when regressors are time-varying. We provide numerical experiments for a variant of Nesterov's provably hard convex optimization problem with time-varying regressors, as well as the problem of recovering an image with a time-varying blur and noise using streaming data.
△ Less
Submitted 4 June, 2021; v1 submitted 4 May, 2020;
originally announced May 2020.
-
Parameter Estimation in Adaptive Control of Time-Varying Systems Under a Range of Excitation Conditions
Authors:
Joseph E. Gaudio,
Anuradha M. Annaswamy,
Eugene Lavretsky,
Michael A. Bolender
Abstract:
This paper presents a new parameter estimation algorithm for the adaptive control of a class of time-varying plants. The main feature of this algorithm is a matrix of time-varying learning rates, which enables parameter estimation error trajectories to tend exponentially fast towards a compact set whenever excitation conditions are satisfied. This algorithm is employed in a large class of problems…
▽ More
This paper presents a new parameter estimation algorithm for the adaptive control of a class of time-varying plants. The main feature of this algorithm is a matrix of time-varying learning rates, which enables parameter estimation error trajectories to tend exponentially fast towards a compact set whenever excitation conditions are satisfied. This algorithm is employed in a large class of problems where unknown parameters are present and are time-varying. It is shown that this algorithm guarantees global boundedness of the state and parameter errors of the system, and avoids an often used filtering approach for constructing key regressor signals. In addition, intervals of time over which these errors tend exponentially fast toward a compact set are provided, both in the presence of finite and persistent excitation. A projection operator is used to ensure the boundedness of the learning rate matrix, as compared to a time-varying forgetting factor. Numerical simulations are provided to complement the theoretical analysis.
△ Less
Submitted 16 November, 2021; v1 submitted 9 November, 2019;
originally announced November 2019.
-
Adaptive Flight Control in the Presence of Limits on Magnitude and Rate
Authors:
Joseph E. Gaudio,
Anuradha M. Annaswamy,
Michael A. Bolender,
Eugene Lavretsky
Abstract:
Input constraints as well as parametric uncertainties must be accounted for in the design of safe control systems. This paper presents an adaptive controller for multiple-input-multiple-output (MIMO) plants with input magnitude and rate saturation in the presence of parametric uncertainties. A filter is introduced in the control path to accommodate the presence of rate limits. An output feedback a…
▽ More
Input constraints as well as parametric uncertainties must be accounted for in the design of safe control systems. This paper presents an adaptive controller for multiple-input-multiple-output (MIMO) plants with input magnitude and rate saturation in the presence of parametric uncertainties. A filter is introduced in the control path to accommodate the presence of rate limits. An output feedback adaptive controller is designed to stabilize the closed loop system even in the presence of this filter. The overall control architecture includes adaptive laws that are modified to account for the magnitude and rate limits. Analytical guarantees of bounded solutions and satisfactory tracking are provided. Three flight control simulations with nonlinear models of the aircraft dynamics are provided to demonstrate the efficacy of the proposed adaptive controller for open loop stable and unstable systems in the presence of uncertainties in the dynamics as well as input magnitude and rate saturation.
△ Less
Submitted 27 July, 2019;
originally announced July 2019.
-
Connections Between Adaptive Control and Optimization in Machine Learning
Authors:
Joseph E. Gaudio,
Travis E. Gibson,
Anuradha M. Annaswamy,
Michael A. Bolender,
Eugene Lavretsky
Abstract:
This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts,…
▽ More
This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts, new intersections and opportunities for improved algorithm analysis are provided. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
-
Provably Correct Learning Algorithms in the Presence of Time-Varying Features Using a Variational Perspective
Authors:
Joseph E. Gaudio,
Travis E. Gibson,
Anuradha M. Annaswamy,
Michael A. Bolender
Abstract:
Features in machine learning problems are often time-varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current higher order accelerated gradient descent methods unstable or weakens their convergence guarantees. Inspired by methods employed in adaptive control, this paper proposes new algorithms for the case when…
▽ More
Features in machine learning problems are often time-varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current higher order accelerated gradient descent methods unstable or weakens their convergence guarantees. Inspired by methods employed in adaptive control, this paper proposes new algorithms for the case when time-varying features are present, and demonstrates provable performance guarantees. In particular, we develop a unified variational perspective within a continuous time algorithm. This variational perspective includes higher order learning concepts and normalization, both of which stem from adaptive control, and allows stability to be established for dynamical machine learning problems where time-varying features are present. These higher order algorithms are also examined for provably correct learning in adaptive control and identification. Simulations are provided to verify the theoretical results.
△ Less
Submitted 27 May, 2019; v1 submitted 11 March, 2019;
originally announced March 2019.
-
Liquid-solid impacts of yield-stress fluids
Authors:
Marc E. Deetjen,
Brendan C. Blackwell,
Joseph E. Gaudio,
Randy H. Ewoldt
Abstract:
This is an entry to the Gallery of Fluid Motion at the 66th annual meeting of the APS-DFD, held November 2013 in Pittsburgh, PA. In this fluid dynamics video we demonstrate distinct features of yield-stress fluid droplets impacting pre-coated surfaces.
This is an entry to the Gallery of Fluid Motion at the 66th annual meeting of the APS-DFD, held November 2013 in Pittsburgh, PA. In this fluid dynamics video we demonstrate distinct features of yield-stress fluid droplets impacting pre-coated surfaces.
△ Less
Submitted 12 October, 2013;
originally announced October 2013.
