-
Regularizing Numerical Extremals Along Singular Arcs: A Lie-Theoretic Approach
Authors:
Arthur Castello Branco de Oliveira,
Milad Siami,
Eduardo D. Sontag
Abstract:
Numerical ``direct'' approaches to time-optimal control often fail to find solutions that are singular in the sense of the Pontryagin Maximum Principle, performing better when searching for saturated (bang-bang) solutions. In previous work by one of the authors, singular solutions were shown to exist for the time-optimal control problem for fully actuated mechanical systems under hard torque const…
▽ More
Numerical ``direct'' approaches to time-optimal control often fail to find solutions that are singular in the sense of the Pontryagin Maximum Principle, performing better when searching for saturated (bang-bang) solutions. In previous work by one of the authors, singular solutions were shown to exist for the time-optimal control problem for fully actuated mechanical systems under hard torque constraints. Explicit formulas, based on a Lie theoretic analysis of the problem, were given for singular segments of trajectories, but the global structure of solutions remains unknown. In this work, we review the aforementioned framework, and show how to effectively combine these formulas with the use of general-purpose optimal control software packages. By using the explicit formula given by the theory in the intervals where the numerical solution enters a singular arc, we not only obtain an algebraic expression for the control in that interval but we are also able to remove artifacts present in the numerical solution. In this way, the best features of numerical algorithms and theory complement each other and provide a better picture of the global optimal structure. We illustrate the technique on a two degree of freedom robotic arm example, using two distinct optimal control numerical software packages running on different programming languages.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
Properties of Immersions for Systems with Multiple Limit Sets with Implications to Learning Koopman Embeddings
Authors:
Zexiang Liu,
Necmiye Ozay,
Eduardo D. Sontag
Abstract:
Linear immersions (or Koopman eigenmap**s) of a nonlinear system have wide applications in prediction and control. In this work, we study the properties of linear immersions for nonlinear systems with multiple omega-limit sets. While previous research has indicated the possibility of discontinuous one-to-one linear immersions for such systems, it has been unclear whether continuous one-to-one li…
▽ More
Linear immersions (or Koopman eigenmap**s) of a nonlinear system have wide applications in prediction and control. In this work, we study the properties of linear immersions for nonlinear systems with multiple omega-limit sets. While previous research has indicated the possibility of discontinuous one-to-one linear immersions for such systems, it has been unclear whether continuous one-to-one linear immersions are attainable. Under mild conditions, we prove that any continuous immersion to a class of systems including linear systems collapses all the omega-limit sets, and thus cannot be one-to-one. Furthermore, we show that this property is also shared by approximate linear immersions learned from data as sample size increases and sampling interval decreases. Multiple examples are studied to illustrate our results.
△ Less
Submitted 20 January, 2024; v1 submitted 28 December, 2023;
originally announced December 2023.
-
Small-Disturbance Input-to-State Stability of Perturbed Gradient Flows: Applications to LQR Problem
Authors:
Leilei Cui,
Zhong-** Jiang,
Eduardo D. Sontag
Abstract:
This paper studies the effect of perturbations on the gradient flow of a general nonlinear programming problem, where the perturbation may arise from inaccurate gradient estimation in the setting of data-driven optimization. Under suitable conditions on the objective function, the perturbed gradient flow is shown to be small-disturbance input-to-state stable (ISS), which implies that, in the prese…
▽ More
This paper studies the effect of perturbations on the gradient flow of a general nonlinear programming problem, where the perturbation may arise from inaccurate gradient estimation in the setting of data-driven optimization. Under suitable conditions on the objective function, the perturbed gradient flow is shown to be small-disturbance input-to-state stable (ISS), which implies that, in the presence of a small-enough perturbation, the trajectories of the perturbed gradient flow must eventually enter a small neighborhood of the optimum. This work was motivated by the question of robustness of direct methods for the linear quadratic regulator problem, and specifically the analysis of the effect of perturbations caused by gradient estimation or round-off errors in policy optimization. We show small-disturbance ISS for three of the most common optimization algorithms: standard gradient flow, natural gradient flow, and Newton gradient flow.
△ Less
Submitted 16 April, 2024; v1 submitted 4 October, 2023;
originally announced October 2023.
-
On the ISS Property of the Gradient Flow for Single Hidden-Layer Neural Networks with Linear Activations
Authors:
Arthur Castello B. de Oliveira,
Milad Siami,
Eduardo D. Sontag
Abstract:
Recent research in neural networks and machine learning suggests that using many more parameters than strictly required by the initial complexity of a regression problem can result in more accurate or faster-converging models -- contrary to classical statistical belief. This phenomenon, sometimes known as ``benign overfitting'', raises questions regarding in what other ways might overparameterizat…
▽ More
Recent research in neural networks and machine learning suggests that using many more parameters than strictly required by the initial complexity of a regression problem can result in more accurate or faster-converging models -- contrary to classical statistical belief. This phenomenon, sometimes known as ``benign overfitting'', raises questions regarding in what other ways might overparameterization affect the properties of a learning problem. In this work, we investigate the effects of overfitting on the robustness of gradient-descent training when subject to uncertainty on the gradient estimation. This uncertainty arises naturally if the gradient is estimated from noisy data or directly measured. Our object of study is a linear neural network with a single, arbitrarily wide, hidden layer and an arbitrary number of inputs and outputs. In this paper we solve the problem for the case where the input and output of our neural-network are one-dimensional, deriving sufficient conditions for robustness of our system based on necessary and sufficient conditions for convergence in the undisturbed case. We then show that the general overparametrized formulation introduces a set of spurious equilibria which lay outside the set where the loss function is minimized, and discuss directions of future work that might extend our current results for more general formulations.
△ Less
Submitted 16 May, 2023;
originally announced May 2023.
-
Remarks on input to state stability of perturbed gradient flows, motivated by model-free feedback control learning
Authors:
Eduardo D. Sontag
Abstract:
Recent work on data-driven control and reinforcement learning has renewed interest in a relative old field in control theory: model-free optimal control approaches which work directly with a cost function and do not rely upon perfect knowledge of a system model. Instead, an "oracle" returns an estimate of the cost associated to, for example, a proposed linear feedback law to solve a linear-quadrat…
▽ More
Recent work on data-driven control and reinforcement learning has renewed interest in a relative old field in control theory: model-free optimal control approaches which work directly with a cost function and do not rely upon perfect knowledge of a system model. Instead, an "oracle" returns an estimate of the cost associated to, for example, a proposed linear feedback law to solve a linear-quadratic regulator problem. This estimate, and an estimate of the gradient of the cost, might be obtained by performing experiments on the physical system being controlled. This motivates in turn the analysis of steepest descent algorithms and their associated gradient differential equations. This note studies the effect of errors in the estimation of the gradient, framed in the language of input to state stability, where the input represents a perturbation from the true gradient. Since one needs to study systems evolving on proper open subsets of Euclidean space, a self-contained review of input to state stability definitions and theorems for systems that evolve on such sets is included. The results are then applied to the study of noisy gradient systems, as well as the associated steepest descent algorithms.
△ Less
Submitted 28 August, 2021; v1 submitted 5 August, 2021;
originally announced August 2021.
-
Edge Selection in Bilinear Dynamical Networks
Authors:
Arthur Castello B. de Oliveira,
Milad Siami,
Eduardo D. Sontag
Abstract:
We develop some basic principles for the design and robustness analysis of a continuous-time bilinear dynamical network, where an attacker can manipulate the strength of the interconnections/edges between some of the agents/nodes. We formulate the edge protection optimization problem of picking a limited number of attack-free edges and minimizing the impact of the attack over the bilinear dynamica…
▽ More
We develop some basic principles for the design and robustness analysis of a continuous-time bilinear dynamical network, where an attacker can manipulate the strength of the interconnections/edges between some of the agents/nodes. We formulate the edge protection optimization problem of picking a limited number of attack-free edges and minimizing the impact of the attack over the bilinear dynamical network.In particular, the $\mathcal{H}_2$-norm of bilinear systems is known to capture robustness and performance properties analogous to its linear counterpart and provides valuable insights for identifying which edges are most sensitive to attacks. The exact optimization problem is combinatorial in the number of edges, and brute-force approaches show poor scalability. However, we show that the $\mathcal{H}_2$-norm as a cost function is supermodular and, therefore, allows for efficient greedy approximations of the optimal solution. We illustrate and compare the effectiveness of our theoretical findings via numerical simulations.
△ Less
Submitted 20 March, 2023; v1 submitted 8 September, 2020;
originally announced September 2020.
-
A remark on incoherent feedforward circuits as change detectors and feedback controllers
Authors:
Eduardo D. Sontag
Abstract:
This note analyzes incoherent feedforward loops in signal processing and control. It studies the response properties of IFFL's to exponentially growing inputs, both for a standard version of the IFFL and for a variation in which the output variable has a positive self-feedback term. It also considers a negative feedback configuration, using such a device as a controller. It uncovers a somewhat sur…
▽ More
This note analyzes incoherent feedforward loops in signal processing and control. It studies the response properties of IFFL's to exponentially growing inputs, both for a standard version of the IFFL and for a variation in which the output variable has a positive self-feedback term. It also considers a negative feedback configuration, using such a device as a controller. It uncovers a somewhat surprising phenomenon in which stabilization is only possible in disconnected regions of parameter space, as the controlled system's growth rate is varied.
△ Less
Submitted 30 January, 2016;
originally announced February 2016.
-
Some remarks on spatial uniformity of solutions of reaction-diffusion PDE's and a related synchronization problem for ODE's
Authors:
Zahra Aminzare,
Eduardo D. Sontag
Abstract:
In this note, we present a condition which guarantees spatial uniformity for the asymptotic behavior of the solutions of a reaction-diffusion PDE with Neumann boundary conditions in one dimension, using the Jacobian matrix of the reaction term and the first Dirichlet eigenvalue of the Laplacian operator on the given spatial domain. We also derive an analog of this PDE result for the synchronizatio…
▽ More
In this note, we present a condition which guarantees spatial uniformity for the asymptotic behavior of the solutions of a reaction-diffusion PDE with Neumann boundary conditions in one dimension, using the Jacobian matrix of the reaction term and the first Dirichlet eigenvalue of the Laplacian operator on the given spatial domain. We also derive an analog of this PDE result for the synchronization of a network of identical ODE models coupled by diffusion terms.
△ Less
Submitted 19 September, 2014; v1 submitted 26 December, 2013;
originally announced December 2013.
-
Logarithmic Lipschitz norms and diffusion-induced instability
Authors:
Zahra Aminzare,
Eduardo D. Sontag
Abstract:
This paper proves that contractive ordinary differential equation systems remain contractive when diffusion is added. Thus, diffusive instabilities, in the sense of the Turing phenomenon, cannot arise for such systems. An important biochemical system is shown to satisfy the required conditions.
This paper proves that contractive ordinary differential equation systems remain contractive when diffusion is added. Thus, diffusive instabilities, in the sense of the Turing phenomenon, cannot arise for such systems. An important biochemical system is shown to satisfy the required conditions.
△ Less
Submitted 2 August, 2012; v1 submitted 1 August, 2012;
originally announced August 2012.
-
Remarks on random dynamical systems with inputs and outputs and a small-gain theorem for monotone RDS
Authors:
Michael Marcondes de Freitas,
Eduardo D. Sontag
Abstract:
This note introduces a new notion of random dynamical system with inputs and outputs, and sketches a small-gain theorem for monotone systems which generalizes a similar theorem known for deterministic systems.
This note introduces a new notion of random dynamical system with inputs and outputs, and sketches a small-gain theorem for monotone systems which generalizes a similar theorem known for deterministic systems.
△ Less
Submitted 28 December, 2012; v1 submitted 6 July, 2012;
originally announced July 2012.
