-
Maximum Likelihood Identification of Uncontrollable Linear Time-Invariant Models for Offset-Free Control
Authors:
Steven J. Kuntz,
James B. Rawlings
Abstract:
Maximum likelihood identification of linear time-invariant models is a difficult problem because it is, in general, a nonlinear semidefinite program, with semidefinite covariance matrix arguments and semidefinite filter stability constraints. To enforce filter stability, we establish a general theory of closed constraints on the system eigenvalues using LMI regions. To solve the identification pro…
▽ More
Maximum likelihood identification of linear time-invariant models is a difficult problem because it is, in general, a nonlinear semidefinite program, with semidefinite covariance matrix arguments and semidefinite filter stability constraints. To enforce filter stability, we establish a general theory of closed constraints on the system eigenvalues using LMI regions. To solve the identification problem, we employ a Cholesky factorization method that reduces the semidefinite program to a standard nonlinear program. Finally, we apply the identification algorithm to a class of linear plant and disturbance models commonly used in offset-free model predictive control applications. Specifically, we consider models that are structured with uncontrollable, integrating disturbance states. We solve this disturbance modeling problem, and validate the resulting controller and estimator performance, in two real-world case studies: first, a low-cost benchmark temperature control laboratory, and second, an industrial-scale chemical reactor at Eastman Chemical's Kingsport plant.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Analysis of the Geometric Structure of Neural Networks and Neural ODEs via Morse Functions
Authors:
Christian Kuehn,
Sara-Viola Kuntz
Abstract:
Besides classical feed-forward neural networks, also neural ordinary differential equations (neural ODEs) gained particular interest in recent years. Neural ODEs can be interpreted as an infinite depth limit of feed-forward or residual neural networks. We study the input-output dynamics of finite and infinite depth neural networks with scalar output. In the finite depth case, the input is a state…
▽ More
Besides classical feed-forward neural networks, also neural ordinary differential equations (neural ODEs) gained particular interest in recent years. Neural ODEs can be interpreted as an infinite depth limit of feed-forward or residual neural networks. We study the input-output dynamics of finite and infinite depth neural networks with scalar output. In the finite depth case, the input is a state associated to a finite number of nodes, which maps under multiple non-linear transformations to the state of one output node. In analogy, a neural ODE maps a linear transformation of the input to a linear transformation of its time-$T$ map. We show that depending on the specific structure of the network, the input-output map has different properties regarding the existence and regularity of critical points. These properties can be characterized via Morse functions, which are scalar functions, where every critical point is non-degenerate. We prove that critical points cannot exist, if the dimension of the hidden layer is monotonically decreasing or the dimension of the phase space is smaller or equal to the input dimension. In the case that critical points exist, we classify their regularity depending on the specific architecture of the network. We show that each critical point is non-degenerate, if for finite depth neural networks the underlying graph has no bottleneck, and if for neural ODEs, the linear transformations used have full rank. For each type of architecture, the proven properties are comparable in the finite and in the infinite depth case. The established theorems allow us to formulate results on universal embedding, i.e.\ on the exact representation of maps by neural networks and neural ODEs. Our dynamical systems viewpoint on the geometric structure of the input-output map provides a fundamental understanding, why certain architectures perform better than others.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
Embedding Capabilities of Neural ODEs
Authors:
Christian Kuehn,
Sara-Viola Kuntz
Abstract:
A class of neural networks that gained particular interest in the last years are neural ordinary differential equations (neural ODEs). We study input-output relations of neural ODEs using dynamical systems theory and prove several results about the exact embedding of maps in different neural ODE architectures in low and high dimension. The embedding capability of a neural ODE architecture can be i…
▽ More
A class of neural networks that gained particular interest in the last years are neural ordinary differential equations (neural ODEs). We study input-output relations of neural ODEs using dynamical systems theory and prove several results about the exact embedding of maps in different neural ODE architectures in low and high dimension. The embedding capability of a neural ODE architecture can be increased by adding, for example, a linear layer, or augmenting the phase space. Yet, there is currently no systematic theory available and our work contributes towards this goal by develo** various embedding results as well as identifying situations, where no embedding is possible. The mathematical techniques used include as main components iterative functional equations, Morse functions and suspension flows, as well as several further ideas from analysis. Although practically, mainly universal approximation theorems are used, our geometric dynamical systems viewpoint on universal embedding provides a fundamental understanding, why certain neural ODE architectures perform better than others.
△ Less
Submitted 28 September, 2023; v1 submitted 2 August, 2023;
originally announced August 2023.
-
Geometric Blow-Up for Folded Limit Cycle Manifolds in Three Time-Scale Systems
Authors:
Samuel Jelbart,
Christian Kuehn,
Sara-Viola Kuntz
Abstract:
Geometric singular perturbation theory provides a powerful mathematical framework for the analysis of 'stationary' multiple time-scale systems which possess a critical manifold, i.e. a smooth manifold of steady states for the limiting fast subsystem, particularly when combined with a method of desingularization known as blow-up. The theory for 'oscillatory' multiple time-scale systems which posses…
▽ More
Geometric singular perturbation theory provides a powerful mathematical framework for the analysis of 'stationary' multiple time-scale systems which possess a critical manifold, i.e. a smooth manifold of steady states for the limiting fast subsystem, particularly when combined with a method of desingularization known as blow-up. The theory for 'oscillatory' multiple time-scale systems which possess a limit cycle manifold instead of (or in addition to) a critical manifold is less developed, particularly in the non-normally hyperbolic regime. We use the blow-up method to analyse the global oscillatory transition near a regular folded limit cycle manifold in a class of three time-scale 'semi-oscillatory' systems with two small parameters. The systems considered behave like oscillatory systems as the smallest perturbation parameter tends to zero, and stationary systems as both perturbation parameters tend to zero. The additional time-scale structure is crucial for the applicability of the blow-up method, which cannot be applied directly to the two time-scale oscillatory counterpart of the problem. Our methods allow us to describe the asymptotics and strong contractivity of all solutions which traverse a neighbourhood of the global singularity. Our main results cover a range of different cases with respect to the relative time-scale of the angular dynamics and the parameter drift. We demonstrate the applicability of our results for systems with periodic forcing in the slow equation, in particular for a class of Liénard equations. Finally, we consider a toy model used to study tip** phenomena in climate systems with periodic forcing in the fast equation, which violates the conditions of our main results, in order to demonstrate the applicability of classical (two time-scale) theory for problems of this kind.
△ Less
Submitted 17 November, 2023; v1 submitted 2 August, 2022;
originally announced August 2022.
