-
Data-driven cold starting of good reservoirs
Authors:
Lyudmila Grigoryeva,
Boumediene Hamzi,
Felix P. Kemeth,
Yannis Kevrekidis,
G Manjunath,
Juan-Pablo Ortega,
Matthys J. Steynberg
Abstract:
Using short histories of observations from a dynamical system, a workflow for the post-training initialization of reservoir computing systems is described. This strategy is called cold-starting, and it is based on a map called the starting map, which is determined by an appropriately short history of observations that maps to a unique initial condition in the reservoir space. The time series gener…
▽ More
Using short histories of observations from a dynamical system, a workflow for the post-training initialization of reservoir computing systems is described. This strategy is called cold-starting, and it is based on a map called the starting map, which is determined by an appropriately short history of observations that maps to a unique initial condition in the reservoir space. The time series generated by the reservoir system using that initial state can be used to run the system in autonomous mode, to produce accurate forecasts of the time series under consideration immediately. By utilizing this map, the lengthy "washouts" that are necessary to initialize reservoir systems can be eliminated, enabling the generation of forecasts using any selection of appropriately short histories of the observations.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
Exploring simplicity bias in 1D dynamical systems
Authors:
Kamaludin Dingle,
Mohammad Alaskandarani,
Boumediene Hamzi,
Ard A. Louis
Abstract:
Arguments inspired by algorithmic information theory predict an inverse relation between the probability and complexity of output patterns in a wide range of input-output maps. This phenomenon is known as \emph{simplicity bias}. By viewing the parameters of dynamical systems as inputs, and resulting (digitised) trajectories as outputs, we study simplicity bias in the logistic map, Gauss map, sine…
▽ More
Arguments inspired by algorithmic information theory predict an inverse relation between the probability and complexity of output patterns in a wide range of input-output maps. This phenomenon is known as \emph{simplicity bias}. By viewing the parameters of dynamical systems as inputs, and resulting (digitised) trajectories as outputs, we study simplicity bias in the logistic map, Gauss map, sine map, Bernoulli map, and tent map. We find that the logistic map, Gauss map, and sine map all exhibit simplicity bias upon sampling of map initial values and parameter values, but the Bernoulli map and tent map do not. The simplicity bias upper bound on output pattern probability is used to make \emph{a priori} predictions for the probability of output patterns. In some cases, the predictions are surprisingly accurate, given that almost no details of the underlying dynamical systems are assumed. More generally, we argue that studying probability-complexity relationships may be a useful tool in studying patterns in dynamical systems.
△ Less
Submitted 12 May, 2024; v1 submitted 13 February, 2024;
originally announced March 2024.
-
Simplicity bias, algorithmic probability, and the random logistic map
Authors:
Boumediene Hamzi,
Kamaludin Dingle
Abstract:
Simplicity bias is an intriguing phenomenon prevalent in various input-output maps, characterized by a preference for simpler, more regular, or symmetric outputs. Notably, these maps typically feature high-probability outputs with simple patterns, whereas complex patterns are exponentially less probable. This bias has been extensively examined and attributed to principles derived from algorithmic…
▽ More
Simplicity bias is an intriguing phenomenon prevalent in various input-output maps, characterized by a preference for simpler, more regular, or symmetric outputs. Notably, these maps typically feature high-probability outputs with simple patterns, whereas complex patterns are exponentially less probable. This bias has been extensively examined and attributed to principles derived from algorithmic information theory and algorithmic probability. In a significant advancement, it has been demonstrated that the renowned logistic map and other one-dimensional maps exhibit simplicity bias when conceptualized as input-output systems. Building upon this work, our research delves into the manifestations of simplicity bias within the random logistic map, specifically focusing on scenarios involving additive noise.
We discover that simplicity bias is observable in the random logistic map for specific ranges of $μ$ and noise magnitudes. Additionally, we find that this bias persists even with the introduction of small measurement noise, though it diminishes as noise levels increase. Our studies also revisit the phenomenon of noise-induced chaos, particularly when $μ=3.83$, revealing its characteristics through complexity-probability plots. Intriguingly, we employ the logistic map to illustrate a paradoxical aspect of data analysis: more data adhering to a consistent trend can occasionally lead to \emph{reduced} confidence in extrapolation predictions, challenging conventional wisdom.
We propose that adopting a probability-complexity perspective in analyzing dynamical systems could significantly enrich statistical learning theories related to series prediction and analysis. This approach not only facilitates a deeper understanding of simplicity bias and its implications but also paves the way for novel methodologies in forecasting complex systems behavior.
△ Less
Submitted 8 April, 2024; v1 submitted 31 December, 2023;
originally announced January 2024.
-
Bridging Algorithmic Information Theory and Machine Learning: A New Approach to Kernel Learning
Authors:
Boumediene Hamzi,
Marcus Hutter,
Houman Owhadi
Abstract:
Machine Learning (ML) and Algorithmic Information Theory (AIT) look at Complexity from different points of view. We explore the interface between AIT and Kernel Methods (that are prevalent in ML) by adopting an AIT perspective on the problem of learning kernels from data, in kernel ridge regression, through the method of Sparse Kernel Flows. In particular, by looking at the differences and commona…
▽ More
Machine Learning (ML) and Algorithmic Information Theory (AIT) look at Complexity from different points of view. We explore the interface between AIT and Kernel Methods (that are prevalent in ML) by adopting an AIT perspective on the problem of learning kernels from data, in kernel ridge regression, through the method of Sparse Kernel Flows. In particular, by looking at the differences and commonalities between Minimal Description Length (MDL) and Regularization in Machine Learning (RML), we prove that the method of Sparse Kernel Flows is the natural approach to adopt to learn kernels from data. This approach aligns naturally with the MDL principle, offering a more robust theoretical basis than the existing reliance on cross-validation. The study reveals that deriving Sparse Kernel Flows does not require a statistical approach; instead, one can directly engage with code-lengths and complexities, concepts central to AIT. Thereby, this approach opens the door to reformulating algorithms in machine learning using tools from AIT, with the aim of providing them a more solid theoretical foundation.
△ Less
Submitted 10 April, 2024; v1 submitted 21 November, 2023;
originally announced November 2023.
-
Further elements on hypernormal forms of non-resonant double Hopf singularities
Authors:
Majid Gazor,
Boumediene Hamzi,
Ahmad Shoghi
Abstract:
In this paper, we deal with hypernormal forms of non-resonant double Hopf singularities. We investigate the infinite level normal form classification of such singularities with nonzero radial cubic part. We provide a normal form decomposition of normal form vector fields in terms of planar-rotating and planar-radial vector fields. These facilitate the pattern recognition and analysis of the corres…
▽ More
In this paper, we deal with hypernormal forms of non-resonant double Hopf singularities. We investigate the infinite level normal form classification of such singularities with nonzero radial cubic part. We provide a normal form decomposition of normal form vector fields in terms of planar-rotating and planar-radial vector fields. These facilitate the pattern recognition and analysis of the corresponding generalized homological maps. This paper is the first instance of the normal form classification for generic non-resonant double Hopf singularities without structural symmetry.
△ Less
Submitted 12 April, 2023;
originally announced April 2023.
-
Learning Dynamical Systems from Data: A Simple Cross-Validation Perspective, Part V: Sparse Kernel Flows for 132 Chaotic Dynamical Systems
Authors:
Lu Yang,
Xiuwen Sun,
Boumediene Hamzi,
Houman Owhadi,
Naiming Xie
Abstract:
Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. A simple and interpretable way to learn a dynamical system from data is to interpolate its vector-field with a data-adapted kernel which can be learned by using Kernel Flows. The method of Kernel Flows is a trainable machine learning method that lea…
▽ More
Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. A simple and interpretable way to learn a dynamical system from data is to interpolate its vector-field with a data-adapted kernel which can be learned by using Kernel Flows. The method of Kernel Flows is a trainable machine learning method that learns the optimal parameters of a kernel based on the premise that a kernel is good if there is no significant loss in accuracy if half of the data is used. The objective function could be a short-term prediction or some other objective for other variants of Kernel Flows). However, this method is limited by the choice of the base kernel. In this paper, we introduce the method of \emph{Sparse Kernel Flows } in order to learn the ``best'' kernel by starting from a large dictionary of kernels. It is based on sparsifying a kernel that is a linear combination of elemental kernels. We apply this approach to a library of 132 chaotic systems.
△ Less
Submitted 27 February, 2023; v1 submitted 24 January, 2023;
originally announced January 2023.
-
One-Shot Learning of Stochastic Differential Equations with Data Adapted Kernels
Authors:
Matthieu Darcy,
Boumediene Hamzi,
Giulia Livieri,
Houman Owhadi,
Peyman Tavallali
Abstract:
We consider the problem of learning Stochastic Differential Equations of the form $dX_t = f(X_t)dt+σ(X_t)dW_t $ from one sample trajectory. This problem is more challenging than learning deterministic dynamical systems because one sample trajectory only provides indirect information on the unknown functions $f$, $σ$, and stochastic process $dW_t$ representing the drift, the diffusion, and the stoc…
▽ More
We consider the problem of learning Stochastic Differential Equations of the form $dX_t = f(X_t)dt+σ(X_t)dW_t $ from one sample trajectory. This problem is more challenging than learning deterministic dynamical systems because one sample trajectory only provides indirect information on the unknown functions $f$, $σ$, and stochastic process $dW_t$ representing the drift, the diffusion, and the stochastic forcing terms, respectively. We propose a method that combines Computational Graph Completion and data adapted kernels learned via a new variant of cross validation. Our approach can be decomposed as follows: (1) Represent the time-increment map $X_t \rightarrow X_{t+dt}$ as a Computational Graph in which $f$, $σ$ and $dW_t$ appear as unknown functions and random variables. (2) Complete the graph (approximate unknown functions and random variables) via Maximum a Posteriori Estimation (given the data) with Gaussian Process (GP) priors on the unknown functions. (3) Learn the covariance functions (kernels) of the GP priors from data with randomized cross-validation. Numerical experiments illustrate the efficacy, robustness, and scope of our method.
△ Less
Submitted 1 December, 2022; v1 submitted 24 September, 2022;
originally announced September 2022.
-
A Note on A Priori Forecasting and Simplicity Bias in Time Series
Authors:
Kamaludin Dingle,
Rafiq Kamal,
Boumediene Hamzi
Abstract:
To what extent can we forecast a time series without fitting to historical data? Can universal patterns of probability help in this task? Deep relations between pattern Kolmogorov complexity and pattern probability have recently been used to make \emph{a priori} probability predictions in a variety of systems in physics, biology and engineering. Here we study \emph{simplicity bias} (SB) -- an expo…
▽ More
To what extent can we forecast a time series without fitting to historical data? Can universal patterns of probability help in this task? Deep relations between pattern Kolmogorov complexity and pattern probability have recently been used to make \emph{a priori} probability predictions in a variety of systems in physics, biology and engineering. Here we study \emph{simplicity bias} (SB) -- an exponential upper bound decay in pattern probability with increasing complexity -- in discretised time series extracted from the World Bank Open Data collection. We predict upper bounds on the probability of discretised series patterns, without fitting to trends in the data. Thus we perform a kind of `forecasting without training data', predicting time series shape patterns \emph{a priori}, but not the actual numerical value of the series. Additionally we make predictions about which of two discretised series is more likely with accuracy of $\sim$80\%, much higher than a 50\% baseline rate, just by using the complexity of each series. These results point to a promising perspective on practical time series forecasting and integration with machine learning methods.
△ Less
Submitted 17 November, 2022; v1 submitted 10 March, 2022;
originally announced March 2022.
-
Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-Sampled Time Series
Authors:
Jonghyeon Lee,
Edward De Brouwer,
Boumediene Hamzi,
Houman Owhadi
Abstract:
A simple and interpretable way to learn a dynamical system from data is to interpolate its vector-field with a kernel. In particular, this strategy is highly efficient (both in terms of accuracy and complexity) when the kernel is data-adapted using Kernel Flows (KF)~\cite{Owhadi19} (which uses gradient-based optimization to learn a kernel based on the premise that a kernel is good if there is no s…
▽ More
A simple and interpretable way to learn a dynamical system from data is to interpolate its vector-field with a kernel. In particular, this strategy is highly efficient (both in terms of accuracy and complexity) when the kernel is data-adapted using Kernel Flows (KF)~\cite{Owhadi19} (which uses gradient-based optimization to learn a kernel based on the premise that a kernel is good if there is no significant loss in accuracy if half of the data is used for interpolation). Despite its previous successes, this strategy (based on interpolating the vector field driving the dynamical system) breaks down when the observed time series is not regularly sampled in time. In this work, we propose to address this problem by directly approximating the vector field of the dynamical system by incorporating time differences between observations in the (KF) data-adapted kernels. We compare our approach with the classical one over different benchmark dynamical systems and show that it significantly improves the forecasting accuracy while remaining simple, fast, and robust.
△ Less
Submitted 25 November, 2021;
originally announced November 2021.
-
Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods
Authors:
Boumediene Hamzi,
Romit Maulik,
Houman Owhadi
Abstract:
Modeling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems. We show that when the kernel of these emulators is also learned from data (using kernel flows, a variant of cross-validation), then the resulting data-driven models are not only faster than equation-based models but are easi…
▽ More
Modeling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems. We show that when the kernel of these emulators is also learned from data (using kernel flows, a variant of cross-validation), then the resulting data-driven models are not only faster than equation-based models but are easier to train than neural networks such as the long short-term memory neural network. In addition, they are also more accurate and predictive than the latter. When trained on geophysical observational data, for example, the weekly averaged global sea-surface temperature, considerable gains are also observed by the proposed technique in comparison to classical partial differential equation-based models in terms of forecast computational cost and accuracy. When trained on publicly available re-analysis data for the daily temperature of the North-American continent, we see significant improvements over classical baselines such as climatology and persistence-based forecast techniques. Although our experiments concern specific examples, the proposed approach is general, and our results support the viability of kernel methods (with learned kernels) for interpretable and computationally efficient geophysical forecasting for a large diversity of processes.
△ Less
Submitted 9 August, 2021; v1 submitted 13 February, 2021;
originally announced March 2021.
-
Kernel methods for center manifold approximation and a data-based version of the Center Manifold Theorem
Authors:
Bernard Haasdonk,
Boumediene Hamzi,
Gabriele Santin,
Dominik Wittwar
Abstract:
For dynamical systems with a non hyperbolic equilibrium, it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to the equilibrium point and to obtain meaningful predictions of its behavior by analyzing a reduced order system on the so-called center manifold.
Sin…
▽ More
For dynamical systems with a non hyperbolic equilibrium, it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to the equilibrium point and to obtain meaningful predictions of its behavior by analyzing a reduced order system on the so-called center manifold.
Since the center manifold is usually not known, good approximation methods are important as the center manifold theorem states that the stability properties of the origin of the reduced order system are the same as those of the origin of the full order system.
In this work, we establish a data-based version of the center manifold theorem that works by considering an approximation in place of an exact manifold. Also the error between the approximated and the original reduced dynamics are quantified.
We then use an apposite data-based kernel method to construct a suitable approximation of the manifold close to the equilibrium, which is compatible with our general error theory. The data are collected by repeated numerical simulation of the full system by means of a high-accuracy solver, which generates sets of discrete trajectories that are then used as a training set. The method is tested on different examples which show promising performance and good accuracy.
△ Less
Submitted 1 December, 2020;
originally announced December 2020.
-
Entropy for practical stabilization
Authors:
Fritz Colonius,
Boumediene Hamzi
Abstract:
For deterministic continuous time nonlinear control systems, epsilon-practical stabilization entropy and practical stabilization entropy are introduced. Here the rate of attraction is specified by a KL-function. Upper and lower bounds for the diverse entropies are proved, with special attention to exponential KL-functions. The relation to feedbacks is discussed, the linear case and several nonline…
▽ More
For deterministic continuous time nonlinear control systems, epsilon-practical stabilization entropy and practical stabilization entropy are introduced. Here the rate of attraction is specified by a KL-function. Upper and lower bounds for the diverse entropies are proved, with special attention to exponential KL-functions. The relation to feedbacks is discussed, the linear case and several nonlinear examples are analyzed in detail.
△ Less
Submitted 3 February, 2021; v1 submitted 17 September, 2020;
originally announced September 2020.
-
Learning dynamical systems from data: a simple cross-validation perspective
Authors:
Boumediene Hamzi,
Houman Owhadi
Abstract:
Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. We present variants of cross-validation (Kernel Flows \cite{Owhadi19} and its variants based on Maximum Mean Discrepancy and Lyapunov exponents) as simple approaches for learning the kernel used in these emulators.
Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. We present variants of cross-validation (Kernel Flows \cite{Owhadi19} and its variants based on Maximum Mean Discrepancy and Lyapunov exponents) as simple approaches for learning the kernel used in these emulators.
△ Less
Submitted 9 July, 2020;
originally announced July 2020.
-
Kernel-based approximation of the Koopman generator and Schrödinger operator
Authors:
Stefan Klus,
Feliks Nüske,
Boumediene Hamzi
Abstract:
Many dimensionality and model reduction techniques rely on estimating dominant eigenfunctions of associated dynamical operators from data. Important examples include the Koopman operator and its generator, but also the Schrödinger operator. We propose a kernel-based method for the approximation of differential operators in reproducing kernel Hilbert spaces and show how eigenfunctions can be estima…
▽ More
Many dimensionality and model reduction techniques rely on estimating dominant eigenfunctions of associated dynamical operators from data. Important examples include the Koopman operator and its generator, but also the Schrödinger operator. We propose a kernel-based method for the approximation of differential operators in reproducing kernel Hilbert spaces and show how eigenfunctions can be estimated by solving auxiliary matrix eigenvalue problems. The resulting algorithms are applied to molecular dynamics and quantum chemistry examples. Furthermore, we exploit that, under certain conditions, the Schrödinger operator can be transformed into a Kolmogorov backward operator corresponding to a drift-diffusion process and vice versa. This allows us to apply methods developed for the analysis of high-dimensional stochastic differential equations to quantum mechanical systems.
△ Less
Submitted 25 December, 2020; v1 submitted 27 May, 2020;
originally announced May 2020.
-
Dimensionality Reduction of Complex Metastable Systems via Kernel Embeddings of Transition Manifolds
Authors:
Andreas Bittracher,
Stefan Klus,
Boumediene Hamzi,
Péter Koltai,
Christof Schütte
Abstract:
We present a novel kernel-based machine learning algorithm for identifying the low-dimensional geometry of the effective dynamics of high-dimensional multiscale stochastic systems. Recently, the authors developed a mathematical framework for the computation of optimal reaction coordinates of such systems that is based on learning a parametrization of a low-dimensional transition manifold in a cert…
▽ More
We present a novel kernel-based machine learning algorithm for identifying the low-dimensional geometry of the effective dynamics of high-dimensional multiscale stochastic systems. Recently, the authors developed a mathematical framework for the computation of optimal reaction coordinates of such systems that is based on learning a parametrization of a low-dimensional transition manifold in a certain function space. In this article, we enhance this approach by embedding and learning this transition manifold in a reproducing kernel Hilbert space, exploiting the favorable properties of kernel embeddings. Under mild assumptions on the kernel, the manifold structure is shown to be preserved under the embedding, and distortion bounds can be derived. This leads to a more robust and more efficient algorithm compared to previous parametrization approaches.
△ Less
Submitted 3 February, 2020; v1 submitted 18 April, 2019;
originally announced April 2019.
-
A Note on Local Mode-in-State Participation Factors for Nonlinear Systems
Authors:
Boumediene Hamzi,
Eyad Abed
Abstract:
The paper studies an extension to nonlinear systems of a recently proposed approach to the concept of modal participation factors. First, a definition is given for local mode-in-state participation factors for smooth nonlinear autonomous systems. The definition is general, and, unlike in the more traditional approach, the resulting participation measures depend on the assumed uncertainty law gover…
▽ More
The paper studies an extension to nonlinear systems of a recently proposed approach to the concept of modal participation factors. First, a definition is given for local mode-in-state participation factors for smooth nonlinear autonomous systems. The definition is general, and, unlike in the more traditional approach, the resulting participation measures depend on the assumed uncertainty law governing the system initial condition. The work follows Hashlamoun, Hassouneh and Abed (2009) in taking a mathematical expectation (or set-theoretic average) of a modal contribution measure with respect to an assumed uncertain initial state. As in the linear case, it is found that a symmetry assumption on the distribution of the initial state results in a tractable calculation and an explicit and simple formula for mode-in-state participation factors.
△ Less
Submitted 10 November, 2018;
originally announced November 2018.
-
Greedy Kernel Methods for Center Manifold Approximation
Authors:
Bernard Haasdonk,
Boumediene Hamzi,
Gabriele Santin,
Dominik Wittwar
Abstract:
For certain dynamical systems it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to a non-hyperbolic equilibrium point, and to obtain meaningful predictions of its behavior by analyzing a reduced dimensional problem. Since the manifold is usually not known, app…
▽ More
For certain dynamical systems it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to a non-hyperbolic equilibrium point, and to obtain meaningful predictions of its behavior by analyzing a reduced dimensional problem. Since the manifold is usually not known, approximation methods are of great interest to obtain qualitative estimates. In this work, we use a data-based greedy kernel method to construct a suitable approximation of the manifold close to the equilibrium. The data are collected by repeated numerical simulation of the full system by means of a high-accuracy solver, which generates sets of discrete trajectories that are then used to construct a surrogate model of the manifold. The method is tested on different examples which show promising performance and good accuracy.
△ Less
Submitted 30 October, 2018; v1 submitted 26 October, 2018;
originally announced October 2018.
-
A Note on Kernel Methods for Multiscale Systems with Critical Transitions
Authors:
Boumediene Hamzi,
Christian Kuehn,
Sameh Mohamed
Abstract:
We study the maximum mean discrepancy (MMD) in the context of critical transitions modelled by fast-slow stochastic dynamical systems. We establish a new link between the dynamical theory of critical transitions with the statistical aspects of the MMD. In particular, we show that a formal approximation of the MMD near fast subsystem bifurcation points can be computed to leading-order. In particula…
▽ More
We study the maximum mean discrepancy (MMD) in the context of critical transitions modelled by fast-slow stochastic dynamical systems. We establish a new link between the dynamical theory of critical transitions with the statistical aspects of the MMD. In particular, we show that a formal approximation of the MMD near fast subsystem bifurcation points can be computed to leading-order. In particular, this leading order approximation shows that the MMD depends intricately on the fast-slow systems parameters and one can only expect to extract warning signs under rather stringent conditions. However, the MMD turns out to be an excellent binary classifier to detect the change point induced by the critical transition. We cross-validate our results by numerical simulations for a van der Pol-type model.
△ Less
Submitted 25 April, 2018;
originally announced April 2018.
-
Approximation of Lyapunov Functions from Noisy Data
Authors:
Peter Giesl,
Boumediene Hamzi,
Martin Rasmussen,
Kevin N. Webster
Abstract:
Methods have previously been developed for the approximation of Lyapunov functions using radial basis functions. However these methods assume that the evolution equations are known. We consider the problem of approximating a given Lyapunov function using radial basis functions where the evolution equations are not known, but we instead have sampled data which is contaminated with noise. We propose…
▽ More
Methods have previously been developed for the approximation of Lyapunov functions using radial basis functions. However these methods assume that the evolution equations are known. We consider the problem of approximating a given Lyapunov function using radial basis functions where the evolution equations are not known, but we instead have sampled data which is contaminated with noise. We propose an algorithm in which we first approximate the underlying vector field, and use this approximation to then approximate the Lyapunov function. Our approach combines elements of machine learning/statistical learning theory with the existing theory of Lyapunov function approximation. Error estimates are provided for our algorithm.
△ Less
Submitted 7 January, 2016;
originally announced January 2016.
-
Kernel Methods for Linear Discrete-Time Equations
Authors:
Fritz Colonius,
Boumediene Hamzi
Abstract:
Methods from learning theory are used in the state space of linear dynamical and control systems in order to estimate the system matrices. An application to stabilization via algebraic Riccati equations is included. The approach is illustrated via a series of numerical examples.
Methods from learning theory are used in the state space of linear dynamical and control systems in order to estimate the system matrices. An application to stabilization via algebraic Riccati equations is included. The approach is illustrated via a series of numerical examples.
△ Less
Submitted 10 August, 2015; v1 submitted 11 July, 2015;
originally announced July 2015.
-
A characterization of normal forms for control systems
Authors:
Boumediene Hamzi,
Jeroen S. W. Lamb,
Debra Lewis
Abstract:
The study of the behavior of solutions of ODEs often benefits from deciding on a convenient choice of coordinates. This choice of coordinates may be used to "simplify" the functional expressions that appear in the vector field in order that the essential features of the flow of the ODE near a critical point become more evident. In the case of the analysis of an ordinary differential equation in th…
▽ More
The study of the behavior of solutions of ODEs often benefits from deciding on a convenient choice of coordinates. This choice of coordinates may be used to "simplify" the functional expressions that appear in the vector field in order that the essential features of the flow of the ODE near a critical point become more evident. In the case of the analysis of an ordinary differential equation in the neighborhood of an equilibrium point, this naturally leads to the consideration of the possibility to remove the maximum number of terms in the Taylor expansion of the vector field up to a given order. This idea was introduced by H. Poincaré and the "simplified" system is called normal form.
On another side, even though in many textbook treatments the emphasis is on the reduction of the number of monomials in the Taylor expansion, one of the main reasons for the success of normal forms lies in the fact that it allows to analyze a dynamical system based on a simpler form and a simpler form doesn't necessarily mean to remove the maximum number of terms in the Taylor series expansion. This observation, led to introduce the so-called "inner-product normal forms". They are based on properly choosing an inner product that allows to simplify the computations. This inner-product will characterize the space overwhich one performs the Taylor series expansion. The elements in this space are the ones that characterize the normal form. Our goal in this paper is to generalize such an approach to control systems.
△ Less
Submitted 28 March, 2013;
originally announced March 2013.
-
Kernel Methods for the Approximation of Some Key Quantities of Nonlinear Systems
Authors:
Jake Bouvrie,
Boumediene Hamzi
Abstract:
We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems - with a reasonable expectation of success - once the nonlinear system has been mapped into a high or infinite dimensi…
▽ More
We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems - with a reasonable expectation of success - once the nonlinear system has been mapped into a high or infinite dimensional feature space. In particular, we develop computable, non-parametric estimators approximating controllability and observability energy functions for nonlinear systems, and study the ellipsoids they induce. In all cases the relevant quantities are estimated from simulated or observed data. It is then shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system.
△ Less
Submitted 31 March, 2016; v1 submitted 2 April, 2012;
originally announced April 2012.
-
Kernel Methods for the Approximation of Nonlinear Systems
Authors:
Jake Bouvrie,
Boumediene Hamzi
Abstract:
We introduce a data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where balanced truncation may be carried out implicitly. This leads to a nonlinear reduc…
▽ More
We introduce a data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where balanced truncation may be carried out implicitly. This leads to a nonlinear reduction map which can be combined with a representation of the system belonging to a reproducing kernel Hilbert space to give a closed, reduced order dynamical system which captures the essential input-output characteristics of the original model. Empirical simulations illustrating the approach are also provided.
△ Less
Submitted 31 March, 2016; v1 submitted 14 August, 2011;
originally announced August 2011.
-
Balanced Reduction of Nonlinear Control Systems in Reproducing Kernel Hilbert Space
Authors:
Jake Bouvrie,
Boumediene Hamzi
Abstract:
We introduce a novel data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where balanced truncation may be carried out implicitly. This leads to a nonlinear…
▽ More
We introduce a novel data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where balanced truncation may be carried out implicitly. This leads to a nonlinear reduction map which can be combined with a representation of the system belonging to a reproducing kernel Hilbert space to give a closed, reduced order dynamical system which captures the essential input-output characteristics of the original model. Empirical simulations illustrating the approach are also provided.
△ Less
Submitted 12 November, 2010;
originally announced November 2010.
