Showing 1–50 of 74 results for author: Mezić, I

Analytic Extended Dynamic Mode Decomposition

Authors: Alexandre Mauroy, Igor Mezic

Abstract: We aim at develo** an EDMD-type algorithm that captures the spectrum of the Koopman operator defined on a reproducing kernel Hilbert space of analytic functions. Our method relies on an orthogonal projection on polynomial subspaces, which is equivalent to Taylor approximation in a data-driven setting. In the case of dynamics with a hyperbolic equilibrium, the method demonstrates excellent perfor… ▽ More We aim at develo** an EDMD-type algorithm that captures the spectrum of the Koopman operator defined on a reproducing kernel Hilbert space of analytic functions. Our method relies on an orthogonal projection on polynomial subspaces, which is equivalent to Taylor approximation in a data-driven setting. In the case of dynamics with a hyperbolic equilibrium, the method demonstrates excellent performance to capture the lattice structured Koopman spectrum based on the eigenvalues of the linearized system at the equilibrium. Moreover, it yields the Taylor approximation of associated principal eigenfunctions. Since the method preserves the triangular structure of the operator, it does not suffer from spectral pollution and, moreover, arbitrary accuracy on the spectrum can be reached with a fixed finite dimension of the approximation. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Comments: 11 pages

A data driven Koopman-Schur decomposition for computational analysis of nonlinear dynamics

Authors: Zlatko Drmač, Igor Mezić

Abstract: This paper introduces a new theoretical and computational framework for a data driven Koopman mode analysis of nonlinear dynamics. To alleviate the potential problem of ill-conditioned eigenvectors in the existing implementations of the Dynamic Mode Decomposition (DMD) and the Extended Dynamic Mode Decomposition (EDMD), the new method introduces a Koopman-Schur decomposition that is entirely based… ▽ More This paper introduces a new theoretical and computational framework for a data driven Koopman mode analysis of nonlinear dynamics. To alleviate the potential problem of ill-conditioned eigenvectors in the existing implementations of the Dynamic Mode Decomposition (DMD) and the Extended Dynamic Mode Decomposition (EDMD), the new method introduces a Koopman-Schur decomposition that is entirely based on unitary transformations. The analysis in terms of the eigenvectors as modes of a Koopman operator compression is replaced with a modal decomposition in terms of a flag of invariant subspaces that correspond to selected eigenvalues. The main computational tool from the numerical linear algebra is the partial ordered Schur decomposition that provides convenient orthonormal bases for these subspaces. In the case of real data, a real Schur form is used and the computation is based on real orthogonal transformations. The new computational scheme is presented in the framework of the Extended DMD and the kernel trick is used. △ Less

Submitted 25 December, 2023; originally announced December 2023.

Comments: 43 pages, 9 figures

MSC Class: 37N30; 65P99; 37M05; 37M25 ACM Class: G.1.3; G.1.7; G.1.10

Invariant Consistent Dynamic Mode Decomposition

Authors: Gowtham S Seenivasaharagavan, Milan Korda, Hassan Arbabi, Igor Mezić

Abstract: Any deterministic autonomous dynamical system may be globally linearized by its' Koopman operator. This object is typically infinite-dimensional and can be approximated by the so-called Dynamic Mode Decomposition (DMD). In DMD, the central idea is to preserve a fundamental property of the Koopman operator: linearity. This work augments DMD by preserving additional properties like functional relati… ▽ More Any deterministic autonomous dynamical system may be globally linearized by its' Koopman operator. This object is typically infinite-dimensional and can be approximated by the so-called Dynamic Mode Decomposition (DMD). In DMD, the central idea is to preserve a fundamental property of the Koopman operator: linearity. This work augments DMD by preserving additional properties like functional relationships between observables and consistency along geometric invariants. The first set of constraints provides a framework for understanding DMD variants like Higher-order DMD and Affine DMD. The latter set guarantees the estimation of Koopman eigen-functions with eigen-value 1, whose level sets are known to delineate invariant sets. These benefits are realized with only a minimal increase in computational cost, primarily due to the linearity of constraints. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Comments: 23 pages, 5 figures

Trajectory Estimation in Unknown Nonlinear Manifold Using Koopman Operator Theory

Authors: Yanran Wang, Michael J. Banks, Igor Mezic, Takashi Hikihara

Abstract: Formation coordination is a critical aspect of swarm robotics, which involves coordinating the motion and behavior of a group of robots to achieve a specific objective. In formation coordination, the robots must maintain a specific spatial arrangement while in motion. In this paper, we present a leader-follower column formation coordination problem in an unknown, two-dimensional nonlinear manifold… ▽ More Formation coordination is a critical aspect of swarm robotics, which involves coordinating the motion and behavior of a group of robots to achieve a specific objective. In formation coordination, the robots must maintain a specific spatial arrangement while in motion. In this paper, we present a leader-follower column formation coordination problem in an unknown, two-dimensional nonlinear manifold, where we redefining it as a trajectory estimation problem. Leveraging Koopman operator theory and Extended Dynamic Mode Decomposition, we estimate the measurement vectors for the follower agent and guide its nonlinear trajectories. △ Less

Submitted 8 December, 2023; originally announced December 2023.

Koopman Learning with Episodic Memory

Authors: William T. Redman, Dean Huang, Maria Fonoberova, Igor Mezić

Abstract: Koopman operator theory, a data-driven dynamical systems framework, has found significant success in learning models from complex, real-world data sets, enabling state-of-the-art prediction and control. The greater interpretability and lower computational costs of these models, compared to traditional machine learning methodologies, make Koopman learning an especially appealing approach. Despite t… ▽ More Koopman operator theory, a data-driven dynamical systems framework, has found significant success in learning models from complex, real-world data sets, enabling state-of-the-art prediction and control. The greater interpretability and lower computational costs of these models, compared to traditional machine learning methodologies, make Koopman learning an especially appealing approach. Despite this, little work has been performed on endowing Koopman learning with the ability to learn from its own mistakes. To address this, we equip Koopman methods - developed for predicting non-stationary time-series - with an episodic memory mechanism, enabling global recall of (or attention to) periods in time where similar dynamics previously occurred. We find that a basic implementation of Koopman learning with episodic memory leads to significant improvements in prediction on synthetic and real-world data. Our framework has considerable potential for expansion, allowing for future advances, and opens exciting new directions for Koopman learning. △ Less

Submitted 21 November, 2023; originally announced November 2023.

Comments: 12 pages, 5 figures

Operator is the Model

Authors: Igor Mezić

Abstract: Koopman operator based models emerged as the leading methodology for machine learning of dynamical systems. But their scope is much larger. In fact they present a new take on modeling of physical systems, and even language. In this article I present some of the underlying mathematical structures, applications, connections to other methodologies such as transformer architectures Koopman operator based models emerged as the leading methodology for machine learning of dynamical systems. But their scope is much larger. In fact they present a new take on modeling of physical systems, and even language. In this article I present some of the underlying mathematical structures, applications, connections to other methodologies such as transformer architectures △ Less

Submitted 17 December, 2023; v1 submitted 27 October, 2023; originally announced October 2023.

On Higher Order Drift and Diffusion Estimates for Stochastic SINDy

Authors: Mathias Wanner, Igor Mezić

Abstract: The Sparse Identification of Nonlinear Dynamics (SINDy) algorithm can be applied to stochastic differential equations to estimate the drift and the diffusion function using data from a realization of the SDE. The SINDy algorithm requires sample data from each of these functions, which is typically estimated numerically from the data of the state. We analyze the performance of the previously propos… ▽ More The Sparse Identification of Nonlinear Dynamics (SINDy) algorithm can be applied to stochastic differential equations to estimate the drift and the diffusion function using data from a realization of the SDE. The SINDy algorithm requires sample data from each of these functions, which is typically estimated numerically from the data of the state. We analyze the performance of the previously proposed estimates for the drift and diffusion function to give bounds on the error for finite data. However, since this algorithm only converges as both the sampling frequency and the length of trajectory go to infinity, obtaining approximations within a certain tolerance may be infeasible. To combat this, we develop estimates with higher orders of accuracy for use in the SINDy framework. For a given sampling frequency, these estimates give more accurate approximations of the drift and diffusion functions, making SINDy a far more feasible system identification method. △ Less

Submitted 25 January, 2024; v1 submitted 30 June, 2023; originally announced June 2023.

Comments: 25 pages, 6 figures and 1 table

MSC Class: 37H99; 37M15; 60H35; 65C40; 93E12

A Koopman Operator-Based Prediction Algorithm and its Application to COVID-19 Pandemic

Authors: Igor Mezic, Zlatko Drmac, Nelida Crnjaric-Zic, Senka Macesic, Maria Fonoberova, Ryan Mohr, Allan Avila, Iva Manojlovic, Aleksandr Andrejcuk

Abstract: The problem of prediction of behavior of dynamical systems has undergone a paradigm shift in the second half of the 20th century with the discovery of the possibility of chaotic dynamics in simple, physical, dynamical systems for which the laws of evolution do not change in time. The essence of the paradigm is the long term exponential divergence of trajectories. However, that paradigm does not ac… ▽ More The problem of prediction of behavior of dynamical systems has undergone a paradigm shift in the second half of the 20th century with the discovery of the possibility of chaotic dynamics in simple, physical, dynamical systems for which the laws of evolution do not change in time. The essence of the paradigm is the long term exponential divergence of trajectories. However, that paradigm does not account for another type of unpredictability: the ``Black Swan" event. It also does not account for the fact that short-term prediction is often possible even in systems with exponential divergence. In our framework, the Black Swan type dynamics occurs when an underlying dynamical system suddenly shifts between dynamics of different types. A learning and prediction system should be capable of recognizing the shift in behavior, exemplified by ``confidence loss". In this paradigm, the predictive power is assessed dynamically and confidence level is used to switch between long term prediction and local-in-time prediction. Here we explore the problem of prediction in systems that exhibit such behavior. The mathematical underpinnings of our theory and algorithms are based on an operator-theoretic approach in which the dynamics of the system are embedded into an infinite-dimensional space. We apply the algorithm to a number of case studies including prediction of influenza cases and the COVID-19 pandemic. The results show that the predictive algorithm is robust to perturbations of the available data, induced for example by delays in reporting or sudden increase in cases due to increase in testing capability. This is achieved in an entirely data-driven fashion, with no underlying mathematical model of the disease. △ Less

Submitted 26 April, 2023; originally announced April 2023.

Identifying Equivalent Training Dynamics

Authors: William T. Redman, Juan M. Bello-Rivas, Maria Fonoberova, Ryan Mohr, Ioannis G. Kevrekidis, Igor Mezić

Abstract: Study of the nonlinear evolution deep neural network (DNN) parameters undergo during training has uncovered regimes of distinct dynamical behavior. While a detailed understanding of these phenomena has the potential to advance improvements in training efficiency and robustness, the lack of methods for identifying when DNN models have equivalent dynamics limits the insight that can be gained from p… ▽ More Study of the nonlinear evolution deep neural network (DNN) parameters undergo during training has uncovered regimes of distinct dynamical behavior. While a detailed understanding of these phenomena has the potential to advance improvements in training efficiency and robustness, the lack of methods for identifying when DNN models have equivalent dynamics limits the insight that can be gained from prior work. Topological conjugacy, a notion from dynamical systems theory, provides a precise definition of dynamical equivalence, offering a possible route to address this need. However, topological conjugacies have historically been challenging to compute. By leveraging advances in Koopman operator theory, we develop a framework for identifying conjugate and non-conjugate training dynamics. To validate our approach, we demonstrate that it can correctly identify a known equivalence between online mirror descent and online gradient descent. We then utilize it to: identify non-conjugate training dynamics between shallow and wide fully connected neural networks; characterize the early phase of training dynamics in convolutional neural networks; uncover non-conjugate training dynamics in Transformers that do and do not undergo grokking. Our results, across a range of DNN architectures, illustrate the flexibility of our framework and highlight its potential for shedding new light on training dynamics. △ Less

Submitted 4 June, 2024; v1 submitted 17 February, 2023; originally announced February 2023.

Comments: 18 pages, 6 figures, 3 supplemental figures

A Transfer Operator Approach to Relativistic Quantum Wavefunction

Authors: Igor Mezic

Abstract: The original intent of the Koopman-von Neumann formalism was to put classical and quantum mechanics on the same footing by introducing an operator formalism into classical mechanics. Here we pursue their path the opposite way and examine what transfer operators can say about quantum mechanical evolution. To that end, we introduce a physically motivated scalar wavefunction formalism for a velocity… ▽ More The original intent of the Koopman-von Neumann formalism was to put classical and quantum mechanics on the same footing by introducing an operator formalism into classical mechanics. Here we pursue their path the opposite way and examine what transfer operators can say about quantum mechanical evolution. To that end, we introduce a physically motivated scalar wavefunction formalism for a velocity field on a 4-dimensional pseudo-Riemannian manifold, and obtain an evolution equation for the associated wavefunction, a generator for an associated weighted transfer operator. The generator of the scalar evolution is of first order in space and time. The probability interpretation of the formalism leads to recovery of the Schrodinger equation in the non-relativistic limit. In the special relativity limit, we show that the scalar wavefunction of Dirac spinors satisfies the new equation. A connection with string theoretic considerations for mass is provided. △ Less

Submitted 23 October, 2022; originally announced October 2022.

Koopman Reduced Order Modeling with Confidence Bounds

Authors: Ryan Mohr, Maria Fonoberova, Igor Mezic

Abstract: This paper introduces a reduced order modeling technique based on Koopman operator theory that gives confidence bounds on the model's predictions. It is based on a data-driven spectral decomposition of said operator. The reduced order model is constructed using a finite number of Koopman eigenvalues and modes while the rest of spectrum is treated as a noise process. This noise process is used to e… ▽ More This paper introduces a reduced order modeling technique based on Koopman operator theory that gives confidence bounds on the model's predictions. It is based on a data-driven spectral decomposition of said operator. The reduced order model is constructed using a finite number of Koopman eigenvalues and modes while the rest of spectrum is treated as a noise process. This noise process is used to extract the confidence bounds. Additionally, we propose a heuristic algorithm to choose the number of deterministic modes to keep in the model. We assume Gaussian observational noise in our models. As the number of modes used for the reduced order model increases, we approach a deterministic plus Gaussian noise model. The Gaussian-ity of the noise can be measured via a Shapiro-Wilk test. As the number of modes increase, the modal noise better approximates a Gaussian distribution. As the number of modes increases past the threshold, the standard deviation of the modal distribution decreases rapidly. This allows us to propose a heuristic algorithm for choosing the number of deterministic modes to keep for the model. △ Less

Submitted 28 September, 2023; v1 submitted 26 September, 2022; originally announced September 2022.

Comments: Extended abstract, updated figures, small changes to text

MSC Class: 47B33; 15A18; 62F25; 62M20

Algorithmic (Semi-)Conjugacy via Koopman Operator Theory

Authors: William T. Redman, Maria Fonoberova, Ryan Mohr, Ioannis G. Kevrekidis, Igor Mezić

Abstract: Iterative algorithms are of utmost importance in decision and control. With an ever growing number of algorithms being developed, distributed, and proprietarized, there is a similarly growing need for methods that can provide classification and comparison. By viewing iterative algorithms as discrete-time dynamical systems, we leverage Koopman operator theory to identify (semi-)conjugacies between… ▽ More Iterative algorithms are of utmost importance in decision and control. With an ever growing number of algorithms being developed, distributed, and proprietarized, there is a similarly growing need for methods that can provide classification and comparison. By viewing iterative algorithms as discrete-time dynamical systems, we leverage Koopman operator theory to identify (semi-)conjugacies between algorithms using their spectral properties. This provides a general framework with which to classify and compare algorithms. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: 6 pages, 5 figures, accepted to IEEE CDC 2022

An Operator Theoretic View on Pruning Deep Neural Networks

Authors: William T. Redman, Maria Fonoberova, Ryan Mohr, Ioannis G. Kevrekidis, Igor Mezic

Abstract: The discovery of sparse subnetworks that are able to perform as well as full models has found broad applied and theoretical interest. While many pruning methods have been developed to this end, the naïve approach of removing parameters based on their magnitude has been found to be as robust as more complex, state-of-the-art algorithms. The lack of theory behind magnitude pruning's success, especia… ▽ More The discovery of sparse subnetworks that are able to perform as well as full models has found broad applied and theoretical interest. While many pruning methods have been developed to this end, the naïve approach of removing parameters based on their magnitude has been found to be as robust as more complex, state-of-the-art algorithms. The lack of theory behind magnitude pruning's success, especially pre-convergence, and its relation to other pruning methods, such as gradient based pruning, are outstanding open questions in the field that are in need of being addressed. We make use of recent advances in dynamical systems theory, namely Koopman operator theory, to define a new class of theoretically motivated pruning algorithms. We show that these algorithms can be equivalent to magnitude and gradient based pruning, unifying these seemingly disparate methods, and find that they can be used to shed light on magnitude pruning's performance during the early part of training. △ Less

Submitted 12 March, 2022; v1 submitted 27 October, 2021; originally announced October 2021.

Comments: 14 pages, 5 figures

Clarifying the effect of mean subtraction on Dynamic Mode Decomposition

Authors: Gowtham S Seenivasaharagavan, Milan Korda, Hassan Arbabi, Igor Mezić

Abstract: Any autonomous nonlinear dynamical system can be viewed as a superposition of infinitely many linear processes, through the so-called Koopman mode decomposition. Its data-driven approximation- Dynamic Mode Decomposition (DMD)- has been extensively developed and deployed across a plethora of fields. In this work, we study the effect of subtracting the temporal mean on the DMD approximation, for obs… ▽ More Any autonomous nonlinear dynamical system can be viewed as a superposition of infinitely many linear processes, through the so-called Koopman mode decomposition. Its data-driven approximation- Dynamic Mode Decomposition (DMD)- has been extensively developed and deployed across a plethora of fields. In this work, we study the effect of subtracting the temporal mean on the DMD approximation, for observables possessing only a finite number of Koopman modes. Pre-processing time-sequential training data by removing the temporal mean has been a point of contention in the Companion matrix formulation of DMD. This stems from the potential of said pre-processing to render DMD equivalent to a temporal Discrete Fourier Transform (DFT). We prove that this equivalence is impossible when the training data is linearly consistent and the order of the DMD approximation exceeds the number of Koopman modes. Since model order and training set size are synonymous in this variant of DMD, the parity of DMD and DFT can, therefore, be indicative of inadequate training data. △ Less

Submitted 6 June, 2024; v1 submitted 8 May, 2021; originally announced May 2021.

Comments: 39 pages, 7 figures

MSC Class: 37M10; 37N10; 47N20

Predicting the Critical Number of Layers for Hierarchical Support Vector Regression

Authors: Ryan Mohr, Maria Fonoberova, Zlatko Drmač, Iva Manojlović, Igor Mezić

Abstract: Hierarchical support vector regression (HSVR) models a function from data as a linear combination of SVR models at a range of scales, starting at a coarse scale and moving to finer scales as the hierarchy continues. In the original formulation of HSVR, there were no rules for choosing the depth of the model. In this paper, we observe in a number of models a phase transition in the training error -… ▽ More Hierarchical support vector regression (HSVR) models a function from data as a linear combination of SVR models at a range of scales, starting at a coarse scale and moving to finer scales as the hierarchy continues. In the original formulation of HSVR, there were no rules for choosing the depth of the model. In this paper, we observe in a number of models a phase transition in the training error -- the error remains relatively constant as layers are added, until a critical scale is passed, at which point the training error drops close to zero and remains nearly constant for added layers. We introduce a method to predict this critical scale a priori with the prediction based on the support of either a Fourier transform of the data or the Dynamic Mode Decomposition (DMD) spectrum. This allows us to determine the required number of layers prior to training any models. △ Less

Submitted 21 December, 2020; originally announced December 2020.

Comments: 18 pages, 9 figures

MSC Class: 68Q32; 68T05; 68T07; 37M05; 37M10; 37M25 ACM Class: I.5.2; F.2.1; G.1.3; G.1.6

Modeling, Reduction, and Control of a Helically Actuated Inertial Soft Robotic Arm via the Koopman Operator

Authors: David A. Haggerty, Michael J. Banks, Patrick C. Curtis, Igor Mezić, Elliot W. Hawkes

Abstract: Soft robots promise improved safety and capability over rigid robots when deployed in complex, delicate, and dynamic environments. However, the infinite degrees of freedom and highly nonlinear dynamics of these systems severely complicate their modeling and control. As a step toward addressing this open challenge, we apply the data-driven, Hankel Dynamic Mode Decomposition (HDMD) with time delay o… ▽ More Soft robots promise improved safety and capability over rigid robots when deployed in complex, delicate, and dynamic environments. However, the infinite degrees of freedom and highly nonlinear dynamics of these systems severely complicate their modeling and control. As a step toward addressing this open challenge, we apply the data-driven, Hankel Dynamic Mode Decomposition (HDMD) with time delay observables to the model identification of a highly inertial, helical soft robotic arm with a high number of underactuated degrees of freedom. The resulting model is linear and hence amenable to control via a Linear Quadratic Regulator (LQR). Using our test bed device, a dynamic, lightweight pneumatic fabric arm with an inertial mass at the tip, we show that the combination of HDMD and LQR allows us to command our robot to achieve arbitrary poses using only open loop control. We further show that Koopman spectral analysis gives us a dimensionally reduced basis of modes which decreases computational complexity without sacrificing predictive power. △ Less

Submitted 16 November, 2020; originally announced November 2020.

Comments: Submitted to IEEE International Conference on Robotics and Automation, 2021

Robust Approximation of the Stochastic Koopman Operator

Authors: Mathias Wanner, Igor Mezić

Abstract: We analyze the performance of Dynamic Mode Decomposition (DMD)-based approximations of the stochastic Koopman operator for random dynamical systems where either the dynamics or observables are affected by noise. For many DMD algorithms, the presence of noise can introduce a bias in the DMD operator, leading to poor approximations of the dynamics. In particular, methods using time delayed observabl… ▽ More We analyze the performance of Dynamic Mode Decomposition (DMD)-based approximations of the stochastic Koopman operator for random dynamical systems where either the dynamics or observables are affected by noise. For many DMD algorithms, the presence of noise can introduce a bias in the DMD operator, leading to poor approximations of the dynamics. In particular, methods using time delayed observables, such as Hankel DMD, are biased when the dynamics are random. We introduce a new, robust DMD algorithm that can approximate the stochastic Koopman operator despite the presence of noise. We then demonstrate how this algorithm can be applied to time delayed observables, which allows us to generate a Krylov subspace from a single observable. This allows us to compute a realization of the stochastic Koopman operator using a single observable measured over a single trajectory. We test the performance of the algorithms over several examples. △ Less

Submitted 3 February, 2022; v1 submitted 30 October, 2020; originally announced November 2020.

Comments: 20 pages, six figures

Koopman Operator, Geometry, and Learning

Authors: Igor Mezic

Abstract: We provide a framework for learning of dynamical systems rooted in the concept of representations and Koopman operators. The interplay between the two leads to the full description of systems that can be represented linearly in a finite dimension, based on the properties of the Koopman operator spectrum. The geometry of state space is connected to the notion of representation, both in the linear c… ▽ More We provide a framework for learning of dynamical systems rooted in the concept of representations and Koopman operators. The interplay between the two leads to the full description of systems that can be represented linearly in a finite dimension, based on the properties of the Koopman operator spectrum. The geometry of state space is connected to the notion of representation, both in the linear case - where it is related to joint level sets of eigenfunctions - and in the nonlinear representation case. As shown here, even nonlinear finite-dimensional representations can be learned using the Koopman operator framework, leading to a new class of representation eigenproblems. The connection to learning using neural networks is given. An extension of the Koopman operator theory to "static" maps between different spaces is provided. The effect of the Koopman operator spectrum on Mori-Zwanzig type representations is discussed. △ Less

Submitted 11 October, 2020; originally announced October 2020.

Koopman Resolvent: A Laplace-Domain Analysis of Nonlinear Autonomous Dynamical Systems

Authors: Yoshihiko Susuki, Alexandre Mauroy, Igor Mezic

Abstract: The motivation of our research is to establish a Laplace-domain theory that provides principles and methodology to analyze and synthesize systems with nonlinear dynamics. A semigroup of composition operators defined for nonlinear autonomous dynamical systems -- the Koopman semigroup and its associated Koopman generator -- plays a central role in this study. We introduce the resolvent of the Koopma… ▽ More The motivation of our research is to establish a Laplace-domain theory that provides principles and methodology to analyze and synthesize systems with nonlinear dynamics. A semigroup of composition operators defined for nonlinear autonomous dynamical systems -- the Koopman semigroup and its associated Koopman generator -- plays a central role in this study. We introduce the resolvent of the Koopman generator, which we call the Koopman resolvent, and provide its spectral characterization for three types of nonlinear dynamics: ergodic evolution on an attractor, convergence to a stable equilibrium point, and convergence to a (quasi-)stable limit cycle. This shows that the Koopman resolvent provides the Laplace-domain representation of such nonlinear autonomous dynamics. A computational aspect of the Laplace-domain representation is also discussed with emphasis on non-stationary Koopman modes. △ Less

Submitted 5 May, 2021; v1 submitted 24 September, 2020; originally announced September 2020.

Comments: 24 pages, 2 figures

Journal ref: SIAM Journal on Applied Dynamical Systems, vol.20, no.4, pp.2013-2036, October 2021

On Numerical Approximations of the Koopman Operator

Authors: Igor Mezic

Abstract: We study numerical approaches to computation of spectral properties of composition operators. We provide a characterization of Koopman Modes in Banach spaces using Generalized Laplace Analysis. We cast the Dynamic Mode-Decomposition type methods in the context of Finite Section theory of infinite dimensional operators, and provide an example of a mixing map for which the finite section method fail… ▽ More We study numerical approaches to computation of spectral properties of composition operators. We provide a characterization of Koopman Modes in Banach spaces using Generalized Laplace Analysis. We cast the Dynamic Mode-Decomposition type methods in the context of Finite Section theory of infinite dimensional operators, and provide an example of a mixing map for which the finite section method fails. Under assumptions on the underlying dynamics, we provide the first result on the convergence rate under sample size increase in the finite-section approximation. We study the error in the Krylov subspace version of the finite section method and prove convergence in pseudospectral sense for operators with pure point spectrum. This result indicates that Krylov sequence-based approximations can have low error without an exponential-in-dimension increase in the number of functions needed for approximation. △ Less

Submitted 12 September, 2020; originally announced September 2020.

From unsteady to quasi-steady dynamics in the streamwise-oscillating cylinder wake

Authors: Maysam Shamai, Scott T. M. Dawson, Igor Mezić, Beverley J. McKeon

Abstract: The flow around a cylinder oscillating in the streamwise direction with a frequency, $f_f$, much lower than the shedding frequency, $f_s$, has been relatively less studied than the case when these frequencies have the same order of magnitude, or the transverse oscillation configuration. In this study, Particle Image Velocimetry and Koopman Mode Decomposition are used to investigate the streamwise-… ▽ More The flow around a cylinder oscillating in the streamwise direction with a frequency, $f_f$, much lower than the shedding frequency, $f_s$, has been relatively less studied than the case when these frequencies have the same order of magnitude, or the transverse oscillation configuration. In this study, Particle Image Velocimetry and Koopman Mode Decomposition are used to investigate the streamwise-oscillating cylinder wake for forcing frequencies $f_f/f_s \sim 0.04-0.2$ and mean Reynolds number, $Re_0 = 900$. The amplitude of oscillation is such that the instantaneous Reynolds number remains above the critical value for vortex shedding at all times. Characterization of the wake reveals a range of phenomena associated with the interaction of the two frequencies, including modulation of both the amplitude and frequency of the wake structure by the forcing. Koopman analysis reveals a frequency spreading of Koopman modes. A scaling parameter and associated transformation are developed to relate the unsteady, or forced, dynamics of a system to that of a quasi-steady, or unforced, system. For the streamwise-oscillating cylinder, it is shown that this transformation leads to a Koopman Mode Decomposition similar to that of the unforced system. △ Less

Submitted 10 July, 2020; originally announced July 2020.

Applications of Koopman Mode Analysis to Neural Networks

Authors: Iva Manojlović, Maria Fonoberova, Ryan Mohr, Aleksandr Andrejčuk, Zlatko Drmač, Yannis Kevrekidis, Igor Mezić

Abstract: We consider the training process of a neural network as a dynamical system acting on the high-dimensional weight space. Each epoch is an application of the map induced by the optimization algorithm and the loss function. Using this induced map, we can apply observables on the weight space and measure their evolution. The evolution of the observables are given by the Koopman operator associated wit… ▽ More We consider the training process of a neural network as a dynamical system acting on the high-dimensional weight space. Each epoch is an application of the map induced by the optimization algorithm and the loss function. Using this induced map, we can apply observables on the weight space and measure their evolution. The evolution of the observables are given by the Koopman operator associated with the induced dynamical system. We use the spectrum and modes of the Koopman operator to realize the above objectives. Our methods can help to, a priori, determine the network depth; determine if we have a bad initialization of the network weights, allowing a restart before training too long; speeding up the training time. Additionally, our methods help enable noise rejection and improve robustness. We show how the Koopman spectrum can be used to determine the number of layers required for the architecture. Additionally, we show how we can elucidate the convergence versus non-convergence of the training process by monitoring the spectrum, in particular, how the existence of eigenvalues clustering around 1 determines when to terminate the learning process. We also show how using Koopman modes we can selectively prune the network to speed up the training procedure. Finally, we show that incorporating loss functions based on negative Sobolev norms can allow for the reconstruction of a multi-scale signal polluted by very large amounts of noise. △ Less

Submitted 21 June, 2020; originally announced June 2020.

ACM Class: I.2.6

Search strategy in a complex and dynamic environment: the MH370 case

Authors: Stefan Ivić, Bojan Crnković, Hassan Arbabi, Sophie Loire, Patrick Clary, Igor Mezić

Abstract: Search and detection of objects on the ocean surface is a challenging task due to the complexity of the drift dynamics and lack of known optimal solutions for the path of the search agents. This challenge was highlighted by the unsuccessful search for Malaysian Flight 370 (MH370) which disappeared on March 8, 2014. In this paper, we propose an improvement of a search algorithm rooted in the ergodi… ▽ More Search and detection of objects on the ocean surface is a challenging task due to the complexity of the drift dynamics and lack of known optimal solutions for the path of the search agents. This challenge was highlighted by the unsuccessful search for Malaysian Flight 370 (MH370) which disappeared on March 8, 2014. In this paper, we propose an improvement of a search algorithm rooted in the ergodic theory of dynamical systems which can accommodate complex geometries and uncertainties of the drifting search areas on the ocean surface. We illustrate the effectiveness of this algorithm in a computational replication of the conducted search for MH370. In comparison to conventional search methods, the proposed algorithm leads to an order of magnitude improvement in success rate over the time period of the actual search operation. Simulations of the proposed search control also indicate that the initial success rate of finding debris increases in the event of delayed search commencement. This is due to the existence of convergence zones in the search area which leads to local aggregation of debris in those zones and hence reduction of the effective size of the area to be searched. △ Less

Submitted 29 July, 2020; v1 submitted 29 April, 2020; originally announced April 2020.

Comments: 15 pages, 9 figures, improved manuscript version

Spectral Analysis of the Koopman Operator for Partial Differential Equations

Authors: Hiroya Nakao, Igor Mezić

Abstract: We provide an overview of the Koopman operator analysis for a class of partial differential equations describing relaxation of the field variable to a stable stationary state. We introduce Koopman eigenfunctionals of the system and use the notion of conjugacy to develop spectral expansion of the Koopman operator. For linear systems such as the diffusion equation, the Koopman eigenfunctionals can b… ▽ More We provide an overview of the Koopman operator analysis for a class of partial differential equations describing relaxation of the field variable to a stable stationary state. We introduce Koopman eigenfunctionals of the system and use the notion of conjugacy to develop spectral expansion of the Koopman operator. For linear systems such as the diffusion equation, the Koopman eigenfunctionals can be expressed as linear functionals of the field variable. The notion of inertial manifolds is shown to correspond to joint zero level sets of Koopman eigenfunctionals, and the notion of isostables is defined as the level sets of the slowest decaying Koopman eigenfunctional. Linear diffusion equation, nonlinear Burgers equation, and nonlinear phase-diffusion equation are analyzed as examples. △ Less

Submitted 29 November, 2020; v1 submitted 21 April, 2020; originally announced April 2020.

Comments: 15 pages, 4 figures

Journal ref: Chaos 30, 113131 (2020)

Control-Oriented, Data-Driven Models of Thermal Dynamics

Authors: Ljuboslav Boskic, Igor Mezic

Abstract: Energy savings from efficiency methods in individual residential buildings are measured in 10's of dollars, while the energy savings from such measures nationally would amount to 10's of billions of dollars, leading to the "tragedy of the commons" effect. The way out of this situation is via deployment of automated, integrated residential energy systems, that provide the user with a seamless, cost… ▽ More Energy savings from efficiency methods in individual residential buildings are measured in 10's of dollars, while the energy savings from such measures nationally would amount to 10's of billions of dollars, leading to the "tragedy of the commons" effect. The way out of this situation is via deployment of automated, integrated residential energy systems, that provide the user with a seamless, cost effective service leading to improvement of comfort and residential experience. Models are of critical importance in this context, as intelligent operating systems depend on them strongly. However, most of the currently used models of thermal behavior of buildings have high complexity leading to problems and implementation. The complexity also obscures the utilization of well know physical properties of buildings such as the thermal mass. In view of this, we investigate data-driven, simple-to-implement residential environmental models that can serve as the basis for energy saving algorithms in both retrofits and new designs of residential buildings. Despite the nonlinearity of the underlying dynamics, using Koopman operator theory framework in this study we show that a linear second order model embedding, that captures the physics that occur inside a single or multi-zone space does well when compared with data simulated using EnergyPlus. This class of models has low complexity. We show that their parameters have physical significance for the large-scale dynamics of a building and are correlated to concepts such as the thermal mass. We investigate consequences of changing the thermal mass on the energy behavior of a building system and provide best practice design suggestions. △ Less

Submitted 25 September, 2020; v1 submitted 9 December, 2019; originally announced December 2019.

Extended Dynamic Mode Decomposition with Learned Koopman Eigenfunctions for Prediction and Control

Authors: Carl Folkestad, Daniel Pastor, Igor Mezic, Ryan Mohr, Maria Fonoberova, Joel Burdick

Abstract: This paper presents a novel learning framework to construct Koopman eigenfunctions for unknown, nonlinear dynamics using data gathered from experiments. The learning framework can extract spectral information from the full nonlinear dynamics by learning the eigenvalues and eigenfunctions of the associated Koopman operator. We then exploit the learned Koopman eigenfunctions to learn a lifted linear… ▽ More This paper presents a novel learning framework to construct Koopman eigenfunctions for unknown, nonlinear dynamics using data gathered from experiments. The learning framework can extract spectral information from the full nonlinear dynamics by learning the eigenvalues and eigenfunctions of the associated Koopman operator. We then exploit the learned Koopman eigenfunctions to learn a lifted linear state-space model. To the best of our knowledge, our method is the first to utilize Koopman eigenfunctions as lifting functions for EDMD-based methods. We demonstrate the performance of the framework in state prediction and closed loop trajectory tracking of a simulated cart pole system. Our method is able to significantly improve the controller performance while relying on linear control methods to do nonlinear control. △ Less

Submitted 18 March, 2020; v1 submitted 20 November, 2019; originally announced November 2019.

Comments: 2020 American Control Conference

Exponentially decaying modes and long-term prediction of sea ice concentration using Koopman Mode Decomposition

Authors: James Hogg, Maria Fonoberova, Igor Mezic

Abstract: Sea ice cover in the Arctic and Antarctic is an important indicator of changes in the climate, with important environmental, economic and security consequences. The complexity of the spatio-temporal dynamics of sea ice makes it difficult to assess the temporal nature of the changes - e.g. linear or exponential - and their precise geographical loci. In this study, Koopman Mode Decomposition (KMD) w… ▽ More Sea ice cover in the Arctic and Antarctic is an important indicator of changes in the climate, with important environmental, economic and security consequences. The complexity of the spatio-temporal dynamics of sea ice makes it difficult to assess the temporal nature of the changes - e.g. linear or exponential - and their precise geographical loci. In this study, Koopman Mode Decomposition (KMD) was applied to satellite data of sea ice concentration for the northern and southern hemispheres to gain insight into the temporal and spatial dynamics of the sea ice behavior and to predict future sea ice behavior. We discover exponentially decaying spatial modes in both hemispheres and discuss their precise spatial extent, and also perform precise geographic predictions of sea ice concentration up to four years in the future. This data-driven decomposition technique gives insight in spatial and temporal dynamics not apparent in traditional linear approaches. △ Less

Submitted 4 November, 2019; originally announced November 2019.

On Learning Hamiltonian Systems from Data

Authors: Tom Bertalan, Felix Dietrich, Igor Mezić, Ioannis G. Kevrekidis

Abstract: Concise, accurate descriptions of physical systems through their conserved quantities abound in the natural sciences. In data science, however, current research often focuses on regression problems, without routinely incorporating additional assumptions about the system that generated the data. Here, we propose to explore a particular type of underlying structure in the data: Hamiltonian systems,… ▽ More Concise, accurate descriptions of physical systems through their conserved quantities abound in the natural sciences. In data science, however, current research often focuses on regression problems, without routinely incorporating additional assumptions about the system that generated the data. Here, we propose to explore a particular type of underlying structure in the data: Hamiltonian systems, where an "energy" is conserved. Given a collection of observations of such a Hamiltonian system over time, we extract phase space coordinates and a Hamiltonian function of them that acts as the generator of the system dynamics. The approach employs an autoencoder neural network component to estimate the transformation from observations to the phase space of a Hamiltonian system. An additional neural network component is used to approximate the Hamiltonian function on this constructed space, and the two components are trained jointly. As an alternative approach, we also demonstrate the use of Gaussian processes for the estimation of such a Hamiltonian. After two illustrative examples, we extract an underlying phase space as well as the generating Hamiltonian from a collection of movies of a pendulum. The approach is fully data-driven, and does not assume a particular form of the Hamiltonian function. △ Less

Submitted 4 February, 2020; v1 submitted 29 July, 2019; originally announced July 2019.

MSC Class: 70H05

Journal ref: Chaos 29, 121107 (2019)

Spectral analysis of mixing in 2D high-Reynolds flows

Authors: Hassan Arbabi, Igor Mezic

Abstract: We use spectral analysis of Eulerian and Lagrangian dynamics to study the advective mixing in an incompressible 2D bounded cavity flow. A significant property of such a rotational flow at high Reynolds numbers is that mixing in its core is slower than wall-adjacent areas and corner eddies. We explain this property by considering the resonance between frequencies of unsteady motion -- captured by t… ▽ More We use spectral analysis of Eulerian and Lagrangian dynamics to study the advective mixing in an incompressible 2D bounded cavity flow. A significant property of such a rotational flow at high Reynolds numbers is that mixing in its core is slower than wall-adjacent areas and corner eddies. We explain this property by considering the resonance between frequencies of unsteady motion -- captured by the Koopman spectral analysis of the velocity field -- and the circulation frequency of Lagrangian tracers in the mean flow. In high-Reynolds rotational 2D flows, the vorticity in the rotational core is uniformly distributed, which leads to uniform distribution of circulation periods in the mean flow, i.e., the kinematics in the core of mean flow is like rigid-body rotation. When this ``rigid" core is exposed to velocity fluctuations arising from bifurcations at high Reynolds, it shows more resilience toward resonance in Lagrangian motion and hence mixes more slowly compared to other areas. We also show how our qualitative resonance argument extends to chaotic flows where the classical tools of dynamical systems are not applicable. △ Less

Submitted 31 March, 2020; v1 submitted 24 March, 2019; originally announced March 2019.

Comments: Some changes in the presentation

MSC Class: 76F25; 37A25

On least squares problems with certain Vandermonde--Khatri--Rao structure with applications to DMD

Authors: Zlatko Drmač, Igor Mezić, Ryan Mohr

Abstract: This paper proposes a new computational method for solving structured least squares problems that arise in the process of identification of coherent structures in fluid flows. It is deployed in combination with dynamic mode decomposition (DMD) which provides a non-orthogonal set of modes --- corresponding to particular temporal frequencies --- a subset of which is used to represent time snapshots… ▽ More This paper proposes a new computational method for solving structured least squares problems that arise in the process of identification of coherent structures in fluid flows. It is deployed in combination with dynamic mode decomposition (DMD) which provides a non-orthogonal set of modes --- corresponding to particular temporal frequencies --- a subset of which is used to represent time snapshots of the underlying dynamics. The coefficients of the representation are determined from a solution of a structured linear least squares problem with the matrix that involves the Khatri--Rao product of a triangular and a Vandermonde matrix. Such a structure allows a very efficient normal equation based least squares solution, which is used in state of the art CFD tools such as the sparsity promoting DMD (DMDSP). A new numerical analysis of the normal equations approach provides insights about its applicability and its limitations. Relevant condition numbers that determine numerical robustness are identified and discussed. Further, the paper offers a corrected semi-normal solution and QR factorization based algorithms. It is shown how to use the Vandermonde--Khatri--Rao structure to efficiently compute the QR factorization of the least squares coefficient matrix, thus providing a new computational tool for the ill-conditioned cases where the normal equations may fail to compute a sufficiently accurate solution. Altogether, the presented material provides a firm numerical linear algebra framework for a class of structured least squares problems arising in a variety of applications. △ Less

Submitted 29 November, 2018; originally announced November 2018.

Optimal construction of Koopman eigenfunctions for prediction and control

Authors: Milan Korda, Igor Mezić

Abstract: This work presents a novel data-driven framework for constructing eigenfunctions of the Koopman operator geared toward prediction and control. The method leverages the richness of the spectrum of the Koopman operator away from attractors to construct a rich set of eigenfunctions such that the state (or any other observable quantity of interest) is in the span of these eigenfunctions and hence pred… ▽ More This work presents a novel data-driven framework for constructing eigenfunctions of the Koopman operator geared toward prediction and control. The method leverages the richness of the spectrum of the Koopman operator away from attractors to construct a rich set of eigenfunctions such that the state (or any other observable quantity of interest) is in the span of these eigenfunctions and hence predictable in a linear fashion. The eigenfunction construction is optimization-based with no dictionary selection required. Once a predictor for the uncontrolled part of the system is obtained in this way, the incorporation of control is done through a multi-step prediction error minimization, carried out by a simple linear least-squares regression. The predictor so obtained is in the form of a linear controlled dynamical system and can be readily applied within the Koopman model predictive control framework of [12] to control nonlinear dynamical systems using linear model predictive control tools. The method is entirely data-driven and based purely on convex optimization, with no reliance on neural networks or other non-convex machine learning tools. The novel eigenfunction construction method is also analyzed theoretically, proving rigorously that the family of eigenfunctions obtained is rich enough to span the space of all continuous functions. In addition, the method is extended to construct generalized eigenfunctions that also give rise Koopman invariant subspaces and hence can be used for linear prediction. Detailed numerical examples with code available online demonstrate the approach, both for prediction and feedback control. △ Less

Submitted 7 May, 2020; v1 submitted 19 October, 2018; originally announced October 2018.

Data driven Koopman spectral analysis in Vandermonde-Cauchy form via the DFT: numerical method and theoretical insights

Authors: Zlatko Drmač, Igor Mezić, Ryan Mohr

Abstract: The goals and contributions of this paper are twofold. It provides a new computational tool for data driven Koopman spectral analysis by taking up the formidable challenge to develop a numerically robust algorithm by following the natural formulation via the Krylov decomposition with the Frobenius companion matrix, and by using its eigenvectors explicitly -- these are defined as the inverse of the… ▽ More The goals and contributions of this paper are twofold. It provides a new computational tool for data driven Koopman spectral analysis by taking up the formidable challenge to develop a numerically robust algorithm by following the natural formulation via the Krylov decomposition with the Frobenius companion matrix, and by using its eigenvectors explicitly -- these are defined as the inverse of the notoriously ill-conditioned Vandermonde matrix. The key step to curb ill-conditioning is the discrete Fourier transform of the snapshots; in the new representation, the Vandermonde matrix is transformed into a generalized Cauchy matrix, which then allows accurate computation by specially tailored algorithms of numerical linear algebra. The second goal is to shed light on the connection between the formulas for optimal reconstruction weights when reconstructing snapshots using subsets of the computed Koopman modes. It is shown how using a certain weaker form of generalized inverses leads to explicit reconstruction formulas that match the abstract results from Koopman spectral theory, in particular the Generalized Laplace Analysis. △ Less

Submitted 28 August, 2018; originally announced August 2018.

Prandtl-Batchelor theorem for flows with quasi-periodic time dependence

Authors: Hassan Arbabi, Igor Mezić

Abstract: The classical Prandtl-Batchelor theorem (Prandtl 1904; Batchelor 1956) states that in the regions of steady 2D flow where viscous forces are small and streamlines are closed, the vorticity is constant. In this paper, we extend this theorem to recirculating flows with quasi-periodic time dependence using ergodic and geometric analysis of Lagrangian dynamics. In particular, we show that 2D quasi-per… ▽ More The classical Prandtl-Batchelor theorem (Prandtl 1904; Batchelor 1956) states that in the regions of steady 2D flow where viscous forces are small and streamlines are closed, the vorticity is constant. In this paper, we extend this theorem to recirculating flows with quasi-periodic time dependence using ergodic and geometric analysis of Lagrangian dynamics. In particular, we show that 2D quasi-periodic viscous flows, in the limit of zero viscosity, cannot converge to recirculating inviscid flows with non-uniform vorticity distribution. A corollary of this result is that if the vorticity contours form a family of closed curves in a quasi-periodic viscous flow, then at the limit of zero viscosity, vorticity is constant in the area enclosed by those curves at all times. △ Less

Submitted 4 December, 2018; v1 submitted 28 August, 2018; originally announced August 2018.

MSC Class: 76U05; 76D99

Invariant Sets in Quasiperiodically Forced Dynamical Systems

Authors: Yoshihiko Susuki, Igor Mezić

Abstract: This paper addresses structures of state space in quasiperiodically forced dynamical systems. We develop a theory of ergodic partition of state space in a class of measure-preserving and dissipative flows, which is a natural extension of the existing theory for measure-preserving maps. The ergodic partition result is based on eigenspace at eigenvalue 0 of the associated Koopman operator, which is… ▽ More This paper addresses structures of state space in quasiperiodically forced dynamical systems. We develop a theory of ergodic partition of state space in a class of measure-preserving and dissipative flows, which is a natural extension of the existing theory for measure-preserving maps. The ergodic partition result is based on eigenspace at eigenvalue 0 of the associated Koopman operator, which is realized via time-averages of observables, and provides a constructive way to visualize a low-dimensional slice through a high-dimensional invariant set. We apply the result to the systems with a finite number of attractors and show that the time-average of a continuous observable is well-defined and reveals the invariant sets, namely, a finite number of basins of attraction. We provide a characterization of invariant sets in the quasiperiodically forced systems. A theoretical result on uniform boundedness of the invariant sets is presented. The series of theoretical results enables numerical analysis of invariant sets in the quasiperiodically forced systems based on the ergodic partition and time-averages. Using this, we analyze a nonlinear model of complex power grids that represents the short-term swing instability, named the coherent swing instability. We show that our theoretical results can be used to understand stability regions in such complex systems. △ Less

Submitted 3 November, 2019; v1 submitted 24 August, 2018; originally announced August 2018.

Comments: 23 pages, 4 figures

Journal ref: SIAM Journal on Applied Dynamical Systems, vol.19, no.1, pp.329-351, January 2020

Spectral Complexity of Directed Graphs and Application to Structural Decomposition

Authors: Igor Mezić, Vladimir A. Fonoberov, Maria Fonoberova, Tuhin Sahai

Abstract: We introduce a new measure of complexity (called spectral complexity) for directed graphs. We start with splitting of the directed graph into its recurrent and non-recurrent parts. We define the spectral complexity metric in terms of the spectrum of the recurrence matrix (associated with the reccurent part of the graph) and the Wasserstein distance. We show that the total complexity of the graph c… ▽ More We introduce a new measure of complexity (called spectral complexity) for directed graphs. We start with splitting of the directed graph into its recurrent and non-recurrent parts. We define the spectral complexity metric in terms of the spectrum of the recurrence matrix (associated with the reccurent part of the graph) and the Wasserstein distance. We show that the total complexity of the graph can then be defined in terms of the spectral complexity, complexities of individual components and edge weights. The essential property of the spectral complexity metric is that it accounts for directed cycles in the graph. In engineered and software systems, such cycles give rise to sub-system interdependencies and increase risk for unintended consequences through positive feedback loops, instabilities, and infinite execution loops in software. In addition, we present a structural decomposition technique that identifies such cycles using a spectral technique. We show that this decomposition complements the well-known spectral decomposition analysis based on the Fiedler vector. We provide several examples of computation of spectral and total complexities, including the demonstration that the complexity increases monotonically with the average degree of a random graph. We also provide an example of spectral complexity computation for the architecture of a realistic fixed wing aircraft system. △ Less

Submitted 31 October, 2018; v1 submitted 17 August, 2018; originally announced August 2018.

Comments: We added new theoretical results in Section 2 and introduced a new section 2.2 devoted to intuitive and physical explanations of the concepts from the paper

Convex computation of extremal invariant measures of nonlinear dynamical systems and Markov processes

Authors: Milan Korda, Didier Henrion, Igor Mezic

Abstract: We propose a convex-optimization-based framework for computation of invariant measures of polynomial dynamical systems and Markov processes, in discrete and continuous time. The set of all invariant measures is characterized as the feasible set of an infinite-dimensional linear program (LP). The objective functional of this LP is then used to single-out a specific measure (or a class of measures)… ▽ More We propose a convex-optimization-based framework for computation of invariant measures of polynomial dynamical systems and Markov processes, in discrete and continuous time. The set of all invariant measures is characterized as the feasible set of an infinite-dimensional linear program (LP). The objective functional of this LP is then used to single-out a specific measure (or a class of measures) extremal with respect to the selected functional such as physical measures, ergodic measures, atomic measures (corresponding to, e.g., periodic orbits) or measures absolutely continuous w.r.t. to a given measure. The infinite-dimensional LP is then approximated using a standard hierarchy of finite-dimensional semidefinite programming problems (SDPs), the solutions of which are truncated moment sequences, which are then used to reconstruct the measure. In particular, we show how to approximate the support of the measure as well as how to construct a sequence of weakly converging absolutely continuous approximations. The presented framework, where a convex functional is minimized or maximized among all invariant measures, can be seen as a generalization of and a computational method to carry out the so called ergodic optimization, where linear functionals are optimized over the set of invariant measures. Finally, we also describe how the presented framework can be adapted to compute eigenmeasures of the Perron-Frobenius operator. △ Less

Submitted 17 September, 2020; v1 submitted 24 July, 2018; originally announced July 2018.

Report number: RApport LAAS n{\textdegree} 18222

On the approximation of Koopman spectra of measure-preserving flows

Authors: Nithin Govindarajan, Ryan Mohr, Shivkumar Chandrasekaran, Igor Mezić

Abstract: The method of using periodic approximations to compute the spectral decomposition of the Koop- man operator is generalized to the class of measure-preserving flows on compact metric spaces. It is shown that the spectral decomposition of the continuous one-parameter unitary group can be approximated from an intermediate time discretization of the flow. A sufficient condition is established between… ▽ More The method of using periodic approximations to compute the spectral decomposition of the Koop- man operator is generalized to the class of measure-preserving flows on compact metric spaces. It is shown that the spectral decomposition of the continuous one-parameter unitary group can be approximated from an intermediate time discretization of the flow. A sufficient condition is established between the time-discretization of the flow and the spatial discretization of the periodic approximation, so that weak convergence of spectra will occur in the limit. This condition effectively translates to the requirement that the spatial refinements must occur at a faster pace than the temporal refinements. This result is contrasted with the well-known CLF condition of finite difference schemes for advection equations. Numerical results of spectral computations are shown for some benchmark examples of volume-preserving flows. △ Less

Submitted 27 June, 2018; originally announced June 2018.

A data-driven Koopman model predictive control framework for nonlinear flows

Authors: Hassan Arbabi, Milan Korda, Igor Mezic

Abstract: The Koopman operator theory is an increasingly popular formalism of dynamical systems theory which enables analysis and prediction of the nonlinear dynamics from measurement data. Building on the recent development of the Koopman model predictive control framework (Korda and Mezic 2016), we propose a methodology for closed-loop feedback control of nonlinear flows in a fully data-driven and model-f… ▽ More The Koopman operator theory is an increasingly popular formalism of dynamical systems theory which enables analysis and prediction of the nonlinear dynamics from measurement data. Building on the recent development of the Koopman model predictive control framework (Korda and Mezic 2016), we propose a methodology for closed-loop feedback control of nonlinear flows in a fully data-driven and model-free manner. In the first step, we compute a Koopman-linear representation of the control system using a variation of the extended dynamic mode decomposition algorithm and then we apply model predictive control to the constructed linear model. Our methodology handles both full-state and sparse measurement; in the latter case, it incorporates the delay-embedding of the available data into the identification and control processes. We illustrate the application of this methodology on the periodic Burgers' equation and the boundary control of a cavity flow governed by the two-dimensional incompressible Navier-Stokes equations. In both examples the proposed methodology is successful in accomplishing the control tasks with sub-millisecond computation time required for evaluation of the control input in closed-loop, thereby allowing for a real-time deployment. △ Less

Submitted 6 June, 2018; v1 submitted 14 April, 2018; originally announced April 2018.

Comments: The MATLAB implementation of the Koopman-MPC framework for the examples is available at https://github.com/arbabiha/KoopmanMPC_for_flowcontrol

MSC Class: 76B75; 35Q93; 76D55; 76N25

Power grid transient stabilization using Koopman model predictive control

Authors: Milan Korda, Yoshihiko Susuki, Igor Mezić

Abstract: This work addresses the problem of transient stabilization of a power grid, following a destabilizing disturbance. The model considered is the cascade interconnection of seven New England test models with the disturbance (e.g., a powerline failure) occurring in the first grid and propagating forward, emulating a wide-area blackout. We consider a data-driven control framework based on the Koopman o… ▽ More This work addresses the problem of transient stabilization of a power grid, following a destabilizing disturbance. The model considered is the cascade interconnection of seven New England test models with the disturbance (e.g., a powerline failure) occurring in the first grid and propagating forward, emulating a wide-area blackout. We consider a data-driven control framework based on the Koopman operator theory, where a linear predictor, evolving on a higher dimensional (embedded) state-space, is built from observed data and subsequently used within a model predictive control (MPC) framework, allowing for the use of efficient computational tools of linear MPC to control this highly nonlinear dynamical system. △ Less

Submitted 28 March, 2018; originally announced March 2018.

Global computation of phase-amplitude reduction for limit-cycle dynamics

Authors: A. Mauroy, I. Mezic

Abstract: Recent years have witnessed increasing interest to phase-amplitude reduction of limit-cycle dynamics. Adding an amplitude coordinate to the phase coordinate allows to take into account the dynamics transversal to the limit cycle and thereby overcomes the main limitations of classic phase reduction (strong convergence to the limit cycle and weak inputs). While previous studies mostly focus on local… ▽ More Recent years have witnessed increasing interest to phase-amplitude reduction of limit-cycle dynamics. Adding an amplitude coordinate to the phase coordinate allows to take into account the dynamics transversal to the limit cycle and thereby overcomes the main limitations of classic phase reduction (strong convergence to the limit cycle and weak inputs). While previous studies mostly focus on local quantities such as infinitesimal responses, a major and limiting challenge of phase-amplitude reduction is to compute amplitude coordinates globally, in the basin of attraction of the limit cycle. In this paper, we propose a method to compute the full set of phase-amplitude coordinates in the large. Our method is based on the so-called Koopman (composition) operator and aims at computing the eigenfunctions of the operator through Laplace averages (in combination with the harmonic balance method). This yields a forward integration method that is not limited to two-dimensional systems. We illustrate the method by computing the so-called isostables of limit cycles in two, three, and four-dimensional state spaces, as well as their responses to strong external inputs. △ Less

Submitted 21 November, 2018; v1 submitted 20 March, 2018; originally announced March 2018.

Comments: 26 pages

On the approximation of Koopman spectra for measure preserving transformations

Authors: Nithin Govindarajan, Ryan Mohr, Shivkumar Chandrasekaran, Igor Mezić

Abstract: For the class of continuous, measure-preserving automorphisms on compact metric spaces, a procedure is proposed for constructing a sequence of finite-dimensional approximations to the associated Koopman operator on a Hilbert space. These finite-dimensional approximations are obtained from the so-called "periodic approximation" of the underlying automorphism and take the form of permutation operato… ▽ More For the class of continuous, measure-preserving automorphisms on compact metric spaces, a procedure is proposed for constructing a sequence of finite-dimensional approximations to the associated Koopman operator on a Hilbert space. These finite-dimensional approximations are obtained from the so-called "periodic approximation" of the underlying automorphism and take the form of permutation operators. Results are established on how these discretizations approximate the Koopman operator spectrally. Specificaly, it is shown that both the spectral measure and the spectral projectors of these permutation operators converge weakly to their infinite dimensional counterparts. Based on this result, a numerical method is derived for computing the spectra of volume-preserving maps on the unit $m$-torus. The discretized Koopman operator can be constructed from solving a bipartite matching problem with $\mathcal{O}(\tilde{n}^{3m/2})$ time-complexity, where $\tilde{n}$ denotes the gridsize on each dimension. By exploiting the permutation structure of the discretized Koopman operator, it is further shown that the projections and density functions are computable in $\mathcal{O}(m \tilde{n}^{m} \log \tilde{n})$ operations using the FFT algorithm. Our method is illustrated on several classical examples of automorphisms on the torus that contain either a discrete, continuous, or a mixed spectra. In addition, the spectral properties of the Chirikov standard map are examined using our method. △ Less

Submitted 7 December, 2018; v1 submitted 11 March, 2018; originally announced March 2018.

Koopman Operator Spectrum for Random Dynamical Systems

Authors: Nelida Črnjarić-Žic, Senka Maćešić, Igor Mezić

Abstract: In this paper we consider the Koopman operator associated with the discrete and the continuous time random dynamical system (RDS). We provide results that characterize the spectrum and the eigenfunctions of the stochastic Koopman operator associated with different types of linear RDS. Then we consider the RDS for which the associated Koopman operator family is a semigroup, especially those for whi… ▽ More In this paper we consider the Koopman operator associated with the discrete and the continuous time random dynamical system (RDS). We provide results that characterize the spectrum and the eigenfunctions of the stochastic Koopman operator associated with different types of linear RDS. Then we consider the RDS for which the associated Koopman operator family is a semigroup, especially those for which the generator can be determined. We define a stochastic Hankel DMD (sHankel-DMD) algorithm for numerical approximations of the spectral objects (eigenvalues, eigenfunctions) of the stochastic Koopman operator and prove its convergence. We apply the methodology to a variety of examples, revealing objects in spectral expansions of the stochastic Koopman operator and enabling model reduction. △ Less

Submitted 16 January, 2019; v1 submitted 8 November, 2017; originally announced November 2017.

MSC Class: 37H10; 47B33; 37M99; 65P99

Data-driven spectral analysis of the Koopman operator

Authors: Milan Korda, Mihai Putinar, Igor Mezić

Abstract: Starting from measured data, we develop a method to compute the fine structure of the spectrum of the Koopman operator with rigorous convergence guarantees. The method is based on the observation that, in the measure-preserving ergodic setting, the moments of the spectral measure associated to a given observable are computable from a single trajectory of this observable. Having finitely many momen… ▽ More Starting from measured data, we develop a method to compute the fine structure of the spectrum of the Koopman operator with rigorous convergence guarantees. The method is based on the observation that, in the measure-preserving ergodic setting, the moments of the spectral measure associated to a given observable are computable from a single trajectory of this observable. Having finitely many moments available, we use the classical Christoffel-Darboux kernel to separate the atomic and absolutely continuous parts of the spectrum, supported by convergence guarantees as the number of moments tends to infinity. In addition, we propose a technique to detect the singular continuous part of the spectrum as well as two methods to approximate the spectral measure with guaranteed convergence in the weak topology, irrespective of whether the singular continuous part is present or not. The proposed method is simple to implement and readily applicable to large-scale systems since the computational complexity is dominated by inverting an $N\times N$ Hermitian positive-definite Toeplitz matrix, where $N$ is the number of moments, for which efficient and numerically stable algorithms exist; in particular, the complexity of the approach is independent of the dimension of the underlying state-space. We also show how to compute, from measured data, the spectral projection on a given segment of the unit circle, allowing us to obtain a finite-dimensional approximation of the operator that explicitly takes into account the point and continuous parts of the spectrum. Finally, we describe a relationship between the proposed method and the so-called Hankel Dynamic Mode Decomposition, providing new insights into the behavior of the eigenvalues of the Hankel DMD operator. A number of numerical examples illustrate the approach, including a study of the spectrum of the lid-driven two-dimensional cavity flow. △ Less

Submitted 24 August, 2018; v1 submitted 17 October, 2017; originally announced October 2017.

Data driven modal decompositions: analysis and enhancements

Authors: Zlatko Drmač, Igor Mezić, Ryan Mohr

Abstract: The Dynamic Mode Decomposition (DMD) is a tool of trade in computational data driven analysis of fluid flows. More generally, it is a computational device for Koopman spectral analysis of nonlinear dynamical systems, with a plethora of applications in applied sciences and engineering. Its exceptional performance triggered developments of several modifications that make the DMD an attractive method… ▽ More The Dynamic Mode Decomposition (DMD) is a tool of trade in computational data driven analysis of fluid flows. More generally, it is a computational device for Koopman spectral analysis of nonlinear dynamical systems, with a plethora of applications in applied sciences and engineering. Its exceptional performance triggered developments of several modifications that make the DMD an attractive method in data driven framework. This work offers further improvements of the DMD to make it more reliable, and to enhance its functionality. In particular, data driven formula for the residuals allows selection of the Ritz pairs, thus providing more precise spectral information of the underlying Koopman operator, and the well-known technique of refining the Ritz vectors is adapted to data driven scenarios. Further, the DMD is formulated in a more general setting of weighted inner product spaces, and the consequences for numerical computation are discussed in detail. Numerical experiments are used to illustrate the advantages of the proposed method, designated as DDMD_RRR (Refined Rayleigh Ritz Data Driven Modal Decomposition). △ Less

Submitted 8 August, 2017; originally announced August 2017.

Applied Koopman Operator Theory for Power Systems Technology

Authors: Yoshihiko Susuki, Igor Mezic, Fredrik Raak, Takashi Hikihara

Abstract: Koopman operator is a composition operator defined for a dynamical system described by nonlinear differential or difference equation. Although the original system is nonlinear and evolves on a finite-dimensional state space, the Koopman operator itself is linear but infinite-dimensional (evolves on a function space). This linear operator captures the full information of the dynamics described by t… ▽ More Koopman operator is a composition operator defined for a dynamical system described by nonlinear differential or difference equation. Although the original system is nonlinear and evolves on a finite-dimensional state space, the Koopman operator itself is linear but infinite-dimensional (evolves on a function space). This linear operator captures the full information of the dynamics described by the original nonlinear system. In particular, spectral properties of the Koopman operator play a crucial role in analyzing the original system. In the first part of this paper, we review the so-called Koopman operator theory for nonlinear dynamical systems, with emphasis on modal decomposition and computation that are direct to wide applications. Then, in the second part, we present a series of applications of the Koopman operator theory to power systems technology. The applications are established as data-centric methods, namely, how to use massive quantities of data obtained numerically and experimentally, through spectral analysis of the Koopman operator: coherency identification of swings in coupled synchronous generators, precursor diagnostic of instabilities in the coupled swing dynamics, and stability assessment of power systems without any use of mathematical models. Future problems of this research direction are identified in the last concluding part of this paper. △ Less

Submitted 29 September, 2017; v1 submitted 31 May, 2017; originally announced June 2017.

Comments: 31 pages, 11 figures

Journal ref: Nonlinear Theory and Its Applications, IEICE, vol.7, no.4, pp.430-459, October 2016

Koopman Spectrum for Cascaded Systems

Authors: Ryan Mohr, Igor Mezić

Abstract: This paper considers the evolution of Koopman principal eigenfunctions of cascaded dynamical systems. If each component subsystem is asymptotically stable, the matrix norms of the linear parts of the component subsystems are strictly increasing, and the component subsystems have disjoint spectrums, there exist perturbation functions for the initial conditions of each component subsystem such that… ▽ More This paper considers the evolution of Koopman principal eigenfunctions of cascaded dynamical systems. If each component subsystem is asymptotically stable, the matrix norms of the linear parts of the component subsystems are strictly increasing, and the component subsystems have disjoint spectrums, there exist perturbation functions for the initial conditions of each component subsystem such that the orbits of the cascaded system and the decoupled component subsystems have zero asymptotic relative error. This implies that the evolutions are asymptotically equivalent; cascaded compositions of stable systems are stable. These results hold for both cascaded systems with linear component subsystem dynamics and linear coupling terms and nonlinear cascades topologically conjugate to the linear case. We further show that the Koopman principal eigenvalues of each component subsystem are also Koopman eigenvalues of the cascaded system. The corresponding Koopman eigenfunctions of the cascaded system are formed by extending the domain of definition of the component systems' principal eigenfunctions and then composing them with the perturbation function. △ Less

Submitted 31 July, 2017; v1 submitted 18 May, 2017; originally announced May 2017.

Study of dynamics in post-transient flows using Koopman mode decomposition

Authors: Hassan Arbabi, Igor Mezić

Abstract: The Koopman Mode Decomposition (KMD) is a data-analysis technique which is often used to extract the spatio-temporal patterns of complex flows. In this paper, we use KMD to study the dynamics of the lid-driven flow in a two-dimensional square cavity based on theorems related to the spectral theory of the Koopman operator. We adapt two algorithms, from the classical Fourier and power spectral analy… ▽ More The Koopman Mode Decomposition (KMD) is a data-analysis technique which is often used to extract the spatio-temporal patterns of complex flows. In this paper, we use KMD to study the dynamics of the lid-driven flow in a two-dimensional square cavity based on theorems related to the spectral theory of the Koopman operator. We adapt two algorithms, from the classical Fourier and power spectral analysis, to compute the discrete and continuous spectrum of the Koopman operator for the post-transient flows. Properties of the Koopman operator spectrum are linked to the sequence of flow regimes occurring between $Re=10000$ and $Re=30000$, and changing the flow nature from steady to aperiodic. The Koopman eigenfunctions for different flow regimes, including flows with mixed spectra, are constructed using the assumption of ergodicity in the state space. The associated Koopman modes show remarkable robustness even as the temporal nature of the flow is changing substantially. We observe that KMD outperforms the Proper Orthogonal Decomposition in reconstruction of the flows with strong quasi-periodic components.c features are present in the flow. △ Less

Submitted 6 December, 2017; v1 submitted 3 April, 2017; originally announced April 2017.

Comments: fixed missing labels in the last figure

MSC Class: 37N10

Journal ref: Phys. Rev. Fluids 2, 124402 (2017)

Koopman Operator Family Spectrum for Nonautonomous Systems - Part 1

Authors: Senka Maćešić, Nelida Črnjarić-Žic, Igor Mezić

Abstract: For every non-autonomous system, there is the related family of Koopman operators $\mathcal{K}^{(t,t_0)}$, parameterized by the time pair $(t,t_0)$. In this paper we are investigating the time dependency of the spectral properties of the Koopman operator family in the linear non-autonomous case and we propose an algorithm for computation of its spectrum from observed data only. To build this algor… ▽ More For every non-autonomous system, there is the related family of Koopman operators $\mathcal{K}^{(t,t_0)}$, parameterized by the time pair $(t,t_0)$. In this paper we are investigating the time dependency of the spectral properties of the Koopman operator family in the linear non-autonomous case and we propose an algorithm for computation of its spectrum from observed data only. To build this algorithm we use the concept of the fundamental matrix of linear non-autonomous systems and some specific aspects of Arnoldi-like methods. In particular, we use Arnoldi-like methods on local data stencils, we exploit the information contained in the Krylov subspace projection error, and discover limitations in the application of Arnoldi-like methods to cases with continous time dependency. We present results of this data-driven algorithm on various linear non-autonomous systems, hybrid as well as continuous in time. In all the examples comparison with exact eigenvalues and eigenfunctions shows excellent performance of the proposed algorithm. △ Less

Submitted 21 March, 2017; originally announced March 2017.

Comments: 30 pages, 30 figures

Whole-Building Fault Detection: A Scalable Approach Using Spectral Methods

Authors: Michael Georgescu, Sophie Loire, Don Kasper, Igor Mezic

Abstract: In this paper, an extension to rules-based fault detection is demonstrated utilizing properties of the Koopman operator. The Koopman operator is an infinite-dimensional, linear operator that captures nonlinear, finite dimensional dynamics. The definition of the Koopman operator enables algorithms that can evaluate the magnitude and coincidence of time-series data. Using spectral properties of this… ▽ More In this paper, an extension to rules-based fault detection is demonstrated utilizing properties of the Koopman operator. The Koopman operator is an infinite-dimensional, linear operator that captures nonlinear, finite dimensional dynamics. The definition of the Koopman operator enables algorithms that can evaluate the magnitude and coincidence of time-series data. Using spectral properties of this operator, diagnostic rule signals generated from building management system (BMS) trend data can be decomposed into components that allow the capture of device behavior at varying time-scales and to a granular level. As it relates to the implementation of fault detection (FDD), this approach creates additional spatial and temporal characterizations of rule signals providing additional data structure and increasing effectiveness with which classification techniques can be applied to the analysis process. The approach permits a knowledge base to be applied in a similar manner to that of a rules-based approach, but the introduced extensions also facilitate the definition of new kinds of diagnostics and overall provide increased analysis potential. △ Less

Submitted 21 March, 2017; originally announced March 2017.

Comments: Proceedings of the 2017 ASHRAE Winter Meeting - Las Vegas, NV, USA, At Las Vegas, NV, Volume: ASHRAE Transactions

On Convergence of Extended Dynamic Mode Decomposition to the Koopman Operator

Authors: Milan Korda, Igor Mezić

Abstract: Extended Dynamic Mode Decomposition (EDMD) is an algorithm that approximates the action of the Koopman operator on an $N$-dimensional subspace of the space of observables by sampling at $M$ points in the state space. Assuming that the samples are drawn either independently or ergodically from some measure $μ$, it was shown that, in the limit as $M\rightarrow\infty$, the EDMD operator… ▽ More Extended Dynamic Mode Decomposition (EDMD) is an algorithm that approximates the action of the Koopman operator on an $N$-dimensional subspace of the space of observables by sampling at $M$ points in the state space. Assuming that the samples are drawn either independently or ergodically from some measure $μ$, it was shown that, in the limit as $M\rightarrow\infty$, the EDMD operator $\mathcal{K}_{N,M}$ converges to $\mathcal{K}_N$, where $\mathcal{K}_N$ is the $L_2(μ)$-orthogonal projection of the action of the Koopman operator on the finite-dimensional subspace of observables. In this work, we show that, as $N \rightarrow \infty$, the operator $\mathcal{K}_N$ converges in the strong operator topology to the Koopman operator. This in particular implies convergence of the predictions of future values of a given observable over any finite time horizon, a fact important for practical applications such as forecasting, estimation and control. In addition, we show that accumulation points of the spectra of $\mathcal{K}_N$ correspond to the eigenvalues of the Koopman operator with the associated eigenfunctions converging weakly to an eigenfunction of the Koopman operator, provided that the weak limit of eigenfunctions is nonzero. As a by-product, we propose an analytic version of the EDMD algorithm which, under some assumptions, allows one to construct $\mathcal{K}_N$ directly, without the use of sampling. Finally, under additional assumptions, we analyze convergence of $\mathcal{K}_{N,N}$ (i.e., $M=N$), proving convergence, along a subsequence, to weak eigenfunctions (or eigendistributions) related to the eigenmeasures of the Perron-Frobenius operator. No assumptions on the observables belonging to a finite-dimensional invariant subspace of the Koopman operator are required throughout. △ Less

Submitted 20 October, 2017; v1 submitted 14 March, 2017; originally announced March 2017.

Journal ref: Journal of Nonlinear Science, 28(2), 687-710, 2018