Showing 1–50 of 69 results for author: Kutz, J N

Shallow Recurrent Decoder for Reduced Order Modeling of Plasma Dynamics

Authors: J. Nathan Kutz, Maryam Reza, Farbod Faraji, Aaron Knoll

Abstract: Reduced order models are becoming increasingly important for rendering complex and multiscale spatio-temporal dynamics computationally tractable. The computational efficiency of such surrogate models is especially important for design, exhaustive exploration and physical understanding. Plasma simulations, in particular those applied to the study of ${\bf E}\times {\bf B}$ plasma discharges and tec… ▽ More Reduced order models are becoming increasingly important for rendering complex and multiscale spatio-temporal dynamics computationally tractable. The computational efficiency of such surrogate models is especially important for design, exhaustive exploration and physical understanding. Plasma simulations, in particular those applied to the study of ${\bf E}\times {\bf B}$ plasma discharges and technologies, such as Hall thrusters, require substantial computational resources in order to resolve the multidimentional dynamics that span across wide spatial and temporal scales. Although high-fidelity computational tools are available to simulate such systems over limited conditions and in highly simplified geometries, simulations of full-size systems and/or extensive parametric studies over many geometric configurations and under different physical conditions are computationally intractable with conventional numerical tools. Thus, scientific studies and industrially oriented modeling of plasma systems, including the important ${\bf E}\times {\bf B}$ technologies, stand to significantly benefit from reduced order modeling algorithms. We develop a model reduction scheme based upon a {\em Shallow REcurrent Decoder} (SHRED) architecture. The scheme uses a neural network for encoding limited sensor measurements in time (sequence-to-sequence encoding) to full state-space reconstructions via a decoder network. Based upon the theory of separation of variables, the SHRED architecture is capable of (i) reconstructing full spatio-temporal fields with as little as three point sensors, even the fields that are not measured with sensor feeds but that are in dynamic coupling with the measured field, and (ii) forecasting the future state of the system using neural network roll-outs from the trained time encoding model. △ Less

Submitted 20 May, 2024; originally announced May 2024.

Comments: 12 pages, 7 figures

Machine Learning in Viscoelastic Fluids via Energy-Based Kernel Embedding

Authors: Samuel E. Otto, Cassio M. Oishi, Fabio Amaral, Steven L. Brunton, J. Nathan Kutz

Abstract: The ability to measure differences in collected data is of fundamental importance for quantitative science and machine learning, motivating the establishment of metrics grounded in physical principles. In this study, we focus on the development of such metrics for viscoelastic fluid flows governed by a large class of linear and nonlinear stress models. To do this, we introduce a kernel function co… ▽ More The ability to measure differences in collected data is of fundamental importance for quantitative science and machine learning, motivating the establishment of metrics grounded in physical principles. In this study, we focus on the development of such metrics for viscoelastic fluid flows governed by a large class of linear and nonlinear stress models. To do this, we introduce a kernel function corresponding to a given viscoelastic stress model that implicitly embeds flowfield snapshots into a Reproducing Kernel Hilbert Space (RKHS) whose squared norm equals the total mechanical energy. Working implicitly with lifted representations in the RKHS via the kernel function provides natural and unambiguous metrics for distances and angles between flowfields without the need for hyperparameter tuning. Additionally, we present a solution to the preimage problem for our kernels, enabling accurate reconstruction of flowfields from their RKHS representations. Through numerical experiments on an unsteady viscoelastic lid-driven cavity flow, we demonstrate the utility of our kernels for extracting energetically-dominant coherent structures in viscoelastic flows across a range of Reynolds and Weissenberg numbers. Specifically, the features extracted by Kernel Principal Component Analysis (KPCA) of flowfield snapshots using our kernel functions yield reconstructions with superior accuracy in terms of mechanical energy compared to conventional methods such as ordinary Principal Component Analysis (PCA) with naïvely-defined state vectors or KPCA with ad-hoc choices of kernel functions. Our findings underscore the importance of principled choices of metrics in both scientific and machine learning investigations of complex fluid systems. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Optimized Dynamic Mode Decomposition for Reconstruction and Forecasting of Atmospheric Chemistry Data

Authors: Meghana Velegar, Christoph Keller, J. Nathan Kutz

Abstract: We introduce the optimized dynamic mode decomposition algorithm for constructing an adaptive and computationally efficient reduced order model and forecasting tool for global atmospheric chemistry dynamics. By exploiting a low-dimensional set of global spatio-temporal modes, interpretable characterizations of the underlying spatial and temporal scales can be computed. Forecasting is also achieved… ▽ More We introduce the optimized dynamic mode decomposition algorithm for constructing an adaptive and computationally efficient reduced order model and forecasting tool for global atmospheric chemistry dynamics. By exploiting a low-dimensional set of global spatio-temporal modes, interpretable characterizations of the underlying spatial and temporal scales can be computed. Forecasting is also achieved with a linear model that uses a linear superposition of the dominant spatio-temporal features. The DMD method is demonstrated on three months of global chemistry dynamics data, showing its significant performance in computational speed and interpretability. We show that the presented decomposition method successfully extracts known major features of atmospheric chemistry, such as summertime surface pollution and biomass burning activities. Moreover, the DMD algorithm allows for rapid reconstruction of the underlying linear model, which can then easily accommodate non-stationary data and changes in the dynamics. △ Less

Submitted 13 April, 2024; originally announced April 2024.

Comments: 13 pages, 16 figures

Statistical Mechanics of Dynamical System Identification

Authors: Andrei A. Klishin, Joseph Bakarji, J. Nathan Kutz, Krithika Manohar

Abstract: Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanical approach to analyze sparse equation discovery algorithms, which typically balance data fit and parsimony through a trial-and-error selection of hyperparameters. In this framework, statistical mechanics offers tools to analyze the interplay between complexit… ▽ More Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanical approach to analyze sparse equation discovery algorithms, which typically balance data fit and parsimony through a trial-and-error selection of hyperparameters. In this framework, statistical mechanics offers tools to analyze the interplay between complexity and fitness, in analogy to that done between entropy and energy. To establish this analogy, we define the optimization procedure as a two-level Bayesian inference problem that separates variable selection from coefficient values and enables the computation of the posterior parameter distribution in closed form. A key advantage of employing statistical mechanical concepts, such as free energy and the partition function, is in the quantification of uncertainty, especially in in the low-data limit; frequently encountered in real-world applications. As the data volume increases, our approach mirrors the thermodynamic limit, leading to distinct sparsity- and noise-induced phase transitions that delineate correct from incorrect identification. This perspective of sparse equation discovery, is versatile and can be adapted to various other equation discovery algorithms. △ Less

Submitted 3 March, 2024; originally announced March 2024.

Comments: 21 RevTeX page, 9 figures

Data-driven local operator finding for reduced-order modelling of plasma systems: II. Application to parametric dynamics

Authors: Farbod Faraji, Maryam Reza, Aaron Knoll, J. Nathan Kutz

Abstract: Real-world systems often exhibit dynamics influenced by various parameters, either inherent or externally controllable, necessitating models capable of reliably capturing these parametric behaviors. Plasma technologies exemplify such systems. For example, phenomena governing global dynamics in Hall thrusters (a spacecraft propulsion technology) vary with various parameters, such as the "self-susta… ▽ More Real-world systems often exhibit dynamics influenced by various parameters, either inherent or externally controllable, necessitating models capable of reliably capturing these parametric behaviors. Plasma technologies exemplify such systems. For example, phenomena governing global dynamics in Hall thrusters (a spacecraft propulsion technology) vary with various parameters, such as the "self-sustained electric field". In this Part II, following on the introduction of our novel data-driven local operator finding algorithm, Phi Method, in Part I, we showcase the method's effectiveness in learning parametric dynamics to predict system behavior across unseen parameter spaces. We present two adaptations: the "parametric Phi Method" and the "ensemble Phi Method", which are demonstrated through 2D fluid-flow-past-a-cylinder and 1D Hall-thruster-plasma-discharge problems. Comparative evaluation against parametric OPT-DMD in the fluid case demonstrates superior predictive performance of the parametric Phi Method. Across both test cases, parametric and ensemble Phi Method reliably recover governing parametric PDEs and offer accurate predictions over test parameters. Ensemble ROM analysis underscores Phi Method's robust learning of dominant dynamic coefficients with high confidence. △ Less

Submitted 3 March, 2024; originally announced March 2024.

Comments: 24 pages, 17 figures

Data-driven local operator finding for reduced-order modelling of plasma systems: I. Concept and verifications

Authors: Farbod Faraji, Maryam Reza, Aaron Knoll, J. Nathan Kutz

Abstract: Reduced-order plasma models that can efficiently predict plasma behavior across various settings and configurations are highly sought after yet elusive. The demand for such models has surged in the past decade due to their potential to facilitate scientific research and expedite the development of plasma technologies. In line with the advancements in computational power and data-driven methods, we… ▽ More Reduced-order plasma models that can efficiently predict plasma behavior across various settings and configurations are highly sought after yet elusive. The demand for such models has surged in the past decade due to their potential to facilitate scientific research and expedite the development of plasma technologies. In line with the advancements in computational power and data-driven methods, we introduce the "Phi Method" in this two-part article. Part I presents this novel algorithm, which employs constrained regression on a candidate term library informed by numerical discretization schemes to discover discretized systems of differential equations. We demonstrate Phi Method's efficacy in deriving reliable and robust reduced-order models (ROMs) for three test cases: the Lorenz attractor, flow past a cylinder, and a 1D Hall-thruster-representative plasma. Part II will delve into the method's application for parametric dynamics discovery. Our results show that ROMs derived from the Phi Method provide remarkably accurate predictions of systems' behavior, whether derived from steady-state or transient-state data. This underscores the method's potential for transforming plasma system modeling. △ Less

Submitted 3 March, 2024; originally announced March 2024.

Comments: 27 pages, 18 figures

PyDMD: A Python package for robust dynamic mode decomposition

Authors: Sara M. Ichinaga, Francesco Andreuzzi, Nicola Demo, Marco Tezzele, Karl Lapo, Gianluigi Rozza, Steven L. Brunton, J. Nathan Kutz

Abstract: The dynamic mode decomposition (DMD) is a simple and powerful data-driven modeling technique that is capable of revealing coherent spatiotemporal patterns from data. The method's linear algebra-based formulation additionally allows for a variety of optimizations and extensions that make the algorithm practical and viable for real-world data analysis. As a result, DMD has grown to become a leading… ▽ More The dynamic mode decomposition (DMD) is a simple and powerful data-driven modeling technique that is capable of revealing coherent spatiotemporal patterns from data. The method's linear algebra-based formulation additionally allows for a variety of optimizations and extensions that make the algorithm practical and viable for real-world data analysis. As a result, DMD has grown to become a leading method for dynamical system analysis across multiple scientific disciplines. PyDMD is a Python package that implements DMD and several of its major variants. In this work, we expand the PyDMD package to include a number of cutting-edge DMD methods and tools specifically designed to handle dynamics that are noisy, multiscale, parameterized, prohibitively high-dimensional, or even strongly nonlinear. We provide a complete overview of the features available in PyDMD as of version 1.0, along with a brief overview of the theory behind the DMD algorithm, information for developers, tips regarding practical DMD usage, and introductory coding examples. All code is available at https://github.com/PyDMD/PyDMD . △ Less

Submitted 12 February, 2024; originally announced February 2024.

Dynamic Mode Decomposition for data-driven analysis and reduced-order modelling of ExB plasmas: II. dynamics forecasting

Authors: Farbod Faraji, Maryam Reza, Aaron Knoll, J. Nathan Kutz

Abstract: In part I of the article, we demonstrated that a variant of the Dynamic Mode Decomposition (DMD) algorithm based on variable projection optimization, called Optimized DMD (OPT-DMD), enables a robust identification of the dominant spatiotemporally coherent modes underlying the data across various test cases representing different physical parameters in an ExB simulation configuration. As the OPT-DM… ▽ More In part I of the article, we demonstrated that a variant of the Dynamic Mode Decomposition (DMD) algorithm based on variable projection optimization, called Optimized DMD (OPT-DMD), enables a robust identification of the dominant spatiotemporally coherent modes underlying the data across various test cases representing different physical parameters in an ExB simulation configuration. As the OPT-DMD can be constrained to produce stable reduced-order models (ROMs) by construction, in this paper, we extend the application of the OPT-DMD and investigate the capabilities of the linear ROM from this algorithm toward forecasting in time of the plasma dynamics in configurations representative of the radial-azimuthal and axial-azimuthal cross-sections of a Hall thruster and over a range of simulation parameters in each test case. The predictive capacity of the OPT-DMD ROM is assessed primarily in terms of short-term dynamics forecast or, in other words, for large ratios of training-to-test data. However, the utility of the ROM for long-term dynamics forecasting is also presented for an example case in the radial-azimuthal configuration. The model's predictive performance is heterogeneous across various test cases. Nonetheless, a remarkable predictiveness is observed in the test cases that do not exhibit highly transient behaviors. Moreover, in all investigated cases, the error between the ground-truth and the reconstructed data from the OPT-DMD ROM remains bounded over time within both the training and the test window. As a result, despite its limitation in terms of generalized applicability to all plasma conditions, the OPT-DMD is proven as a reliable method to develop low computational cost and highly predictive data-driven reduced-order models in systems with a quasi-periodic global evolution of the plasma state. △ Less

Submitted 25 August, 2023; originally announced August 2023.

Comments: 14 pages, 14 figures

Dynamic Mode Decomposition for data-driven analysis and reduced-order modelling of ExB plasmas: I. Extraction of spatiotemporally coherent patterns

Authors: Farbod Faraji, Maryam Reza, Aaron Knoll, J. Nathan Kutz

Abstract: In this two-part article, we evaluate the utility and the generalizability of the Dynamic Mode Decomposition (DMD) algorithm for data-driven analysis and reduced-order modelling of plasma dynamics in cross-field ExB configurations. The DMD algorithm is an interpretable data-driven method that finds a best-fit linear model describing the time evolution of spatiotemporally coherent structures (patte… ▽ More In this two-part article, we evaluate the utility and the generalizability of the Dynamic Mode Decomposition (DMD) algorithm for data-driven analysis and reduced-order modelling of plasma dynamics in cross-field ExB configurations. The DMD algorithm is an interpretable data-driven method that finds a best-fit linear model describing the time evolution of spatiotemporally coherent structures (patterns) in data. We have applied the DMD to extensive high-fidelity datasets generated using a particle-in-cell (PIC) code based on a cost-efficient reduced-order PIC scheme. In this part, we first provide an overview of the concept of DMD and its underpinning Proper Orthogonal and Singular Value Decomposition methods. Two of the main DMD variants are next introduced. We then present and discuss the results of the DMD application in terms of the identification and extraction of the dominant spatiotemporal modes from high-fidelity data over a range of simulation conditions. We demonstrate that the DMD variant based on variable projection optimization (OPT-DMD) outperforms the basic DMD method in identification of the modes underlying the data, leading to notably more reliable reconstruction of the ground-truth. Furthermore, we show in multiple test cases that the discrete frequency spectrum of OPT-DMD-extracted modes is consistent with the temporal spectrum from the Fast Fourier Transform of the data. This observation implies that the OPT-DMD augments the conventional spectral analyses by being able to uniquely reveal the spatial structure of the dominant modes in the frequency spectra, thus, yielding more accessible, comprehensive information on the spatiotemporal characteristics of the plasma phenomena. △ Less

Submitted 25 August, 2023; originally announced August 2023.

Comments: 21 pages, 16 figues

Nonlinear parametric models of viscoelastic fluid flows

Authors: Cassio M. Oishi, Alan A. Kaptanoglu, J. Nathan Kutz, Steven L. Brunton

Abstract: Reduced-order models have been widely adopted in fluid mechanics, particularly in the context of Newtonian fluid flows. These models offer the ability to predict complex dynamics, such as instabilities and oscillations, at a considerably reduced computational cost. In contrast, the reduced-order modeling of non-Newtonian viscoelastic fluid flows remains relatively unexplored. This work leverag… ▽ More Reduced-order models have been widely adopted in fluid mechanics, particularly in the context of Newtonian fluid flows. These models offer the ability to predict complex dynamics, such as instabilities and oscillations, at a considerably reduced computational cost. In contrast, the reduced-order modeling of non-Newtonian viscoelastic fluid flows remains relatively unexplored. This work leverages the sparse identification of nonlinear dynamics algorithm to develop interpretable reduced-order models for viscoelastic flows. In particular, we explore a benchmark oscillatory viscoelastic flow on the four-roll mill geometry using the classical Oldroyd-B fluid. This flow exemplifies many canonical challenges associated with non-Newtonian flows, including transitions, asymmetries, instabilities, and bifurcations arising from the interplay of viscous and elastic forces, all of which require expensive computations in order to resolve the fast timescales and long transients characteristic of such flows. First, we demonstrate the effectiveness of our data-driven surrogate model to predict the transient evolution and accurately reconstruct the spatial flow field for fixed flow parameters. We then develop a fully parametric, nonlinear model capable of capturing the dynamic variations as a function of the Weissenberg number. While the training data is predominantly concentrated on a limit cycle regime for moderate Wi, we show that the parameterized model can be used to extrapolate, accurately predicting the dominant dynamics in the case of high Weissenberg numbers. The proposed methodology represents an initial step in the field of reduced-order modeling for viscoelastic flows with the potential to be further refined and enhanced for the design, optimization, and control of a wide range of non-Newtonian fluid flows using machine learning and reduced-order modeling techniques. △ Less

Submitted 1 December, 2023; v1 submitted 8 August, 2023; originally announced August 2023.

Data-Induced Interactions of Sparse Sensors

Authors: Andrei A. Klishin, J. Nathan Kutz, Krithika Manohar

Abstract: Large-dimensional empirical data in science and engineering frequently has low-rank structure and can be represented as a combination of just a few eigenmodes. Because of this structure, we can use just a few spatially localized sensor measurements to reconstruct the full state of a complex system. The quality of this reconstruction, especially in the presence of sensor noise, depends significantl… ▽ More Large-dimensional empirical data in science and engineering frequently has low-rank structure and can be represented as a combination of just a few eigenmodes. Because of this structure, we can use just a few spatially localized sensor measurements to reconstruct the full state of a complex system. The quality of this reconstruction, especially in the presence of sensor noise, depends significantly on the spatial configuration of the sensors. Multiple algorithms based on gappy interpolation and QR factorization have been proposed to optimize sensor placement. Here, instead of an algorithm that outputs a singular "optimal" sensor configuration, we take a thermodynamic view to compute the full landscape of sensor interactions induced by the training data. The landscape takes the form of the Ising model in statistical physics, and accounts for both the data variance captured at each sensor location and the crosstalk between sensors. Map** out these data-induced sensor interactions allows combining them with external selection criteria and anticipating sensor replacement impacts. △ Less

Submitted 21 July, 2023; originally announced July 2023.

Comments: 17 RevTeX pages, 10 figures

PyKoopman: A Python Package for Data-Driven Approximation of the Koopman Operator

Authors: Shaowu Pan, Eurika Kaiser, Brian M. de Silva, J. Nathan Kutz, Steven L. Brunton

Abstract: PyKoopman is a Python package for the data-driven approximation of the Koopman operator associated with a dynamical system. The Koopman operator is a principled linear embedding of nonlinear dynamics and facilitates the prediction, estimation, and control of strongly nonlinear dynamics using linear systems theory. In particular, PyKoopman provides tools for data-driven system identification for un… ▽ More PyKoopman is a Python package for the data-driven approximation of the Koopman operator associated with a dynamical system. The Koopman operator is a principled linear embedding of nonlinear dynamics and facilitates the prediction, estimation, and control of strongly nonlinear dynamics using linear systems theory. In particular, PyKoopman provides tools for data-driven system identification for unforced and actuated systems that build on the equation-free dynamic mode decomposition (DMD) and its variants. In this work, we provide a brief description of the mathematical underpinnings of the Koopman operator, an overview and demonstration of the features implemented in PyKoopman (with code examples), practical advice for users, and a list of potential extensions to PyKoopman. Software is available at http://github.com/dynamicslab/pykoopman △ Less

Submitted 22 June, 2023; originally announced June 2023.

Comments: 16 pages

Machine Learning for Partial Differential Equations

Authors: Steven L. Brunton, J. Nathan Kutz

Abstract: Partial differential equations (PDEs) are among the most universal and parsimonious descriptions of natural physical laws, capturing a rich variety of phenomenology and multi-scale physics in a compact and symbolic representation. This review will examine several promising avenues of PDE research that are being advanced by machine learning, including: 1) the discovery of new governing PDEs and coa… ▽ More Partial differential equations (PDEs) are among the most universal and parsimonious descriptions of natural physical laws, capturing a rich variety of phenomenology and multi-scale physics in a compact and symbolic representation. This review will examine several promising avenues of PDE research that are being advanced by machine learning, including: 1) the discovery of new governing PDEs and coarse-grained approximations for complex natural and engineered systems, 2) learning effective coordinate systems and reduced-order models to make PDEs more amenable to analysis, and 3) representing solution operators and improving traditional numerical algorithms. In each of these fields, we summarize key advances, ongoing challenges, and opportunities for further development. △ Less

Submitted 29 March, 2023; originally announced March 2023.

Comments: 16 pages, 6 figures

Universal Dynamics of Damped-Driven Systems: The Logistic Map as a Normal Form for Energy Balance

Authors: J. Nathan Kutz, Aminur Rahman, Megan R. Ebers, James Koch, Jason J. Bramburger

Abstract: Damped-driven systems are ubiquitous in engineering and science. Despite the diversity of physical processes observed in a broad range of applications, the underlying instabilities observed in practice have a universal characterization which is determined by the overall gain and loss curves of a given system. The universal behavior of damped-driven systems can be understood from a geometrical desc… ▽ More Damped-driven systems are ubiquitous in engineering and science. Despite the diversity of physical processes observed in a broad range of applications, the underlying instabilities observed in practice have a universal characterization which is determined by the overall gain and loss curves of a given system. The universal behavior of damped-driven systems can be understood from a geometrical description of the energy balance with a minimal number of assumptions. The assumptions on the energy dynamics are as follows: the energy increases monotonically as a function of increasing gain, and the losses become increasingly larger with increasing energy, i.e. there are many routes for dissipation in the system for large input energy. The intersection of the gain and loss curves define an energy balanced solution. By constructing an iterative map between the loss and gain curves, the dynamics can be shown to be homeomorphic to the logistic map, which exhibits a period doubling cascade to chaos. Indeed, the loss and gain curves allow for a geometrical description of the dynamics through a simple Verhulst diagram (cobweb plot). Thus irrespective of the physics and its complexities, this simple geometrical description dictates the universal set of logistic map instabilities that arise in complex damped-driven systems. More broadly, damped-driven systems are a class of non-equilibrium pattern forming systems which have a canonical set of instabilities that are manifest in practice. △ Less

Submitted 21 November, 2022; originally announced November 2022.

Comments: 26 pages, 31 figures

Pilot-Wave Dynamics: Using Dynamic Mode Decomposition to characterize Bifurcations, Routes to Chaos and Emergent Statistics

Authors: J. Nathan Kutz, Andre Nachbin, Peter J. Baddoo, John W. M. Bush

Abstract: We develop a data-driven characterization of the pilot-wave hydrodynamic system in which a bouncing droplet self-propels along the surface of a vibrating bath. We consider drop motion in a confined one-dimensional geometry, and apply the {\em Dynamic mode decomposition} (DMD) in order to characterize the evolution of the wave field as the bath's vibrational acceleration is increased progressively.… ▽ More We develop a data-driven characterization of the pilot-wave hydrodynamic system in which a bouncing droplet self-propels along the surface of a vibrating bath. We consider drop motion in a confined one-dimensional geometry, and apply the {\em Dynamic mode decomposition} (DMD) in order to characterize the evolution of the wave field as the bath's vibrational acceleration is increased progressively. DMD provides a regression framework for adaptively learning a best-fit linear dynamics model over snapshots of spatio-temporal data. The DMD characterization of the wave field yields a fresh perspective on the bouncing-droplet problem that forges valuable new links with the mathematical machinery of quantum mechanics. Moreover, it provides a low-rank characterization of the bifurcation structure of the pilot wave physics. Specifically, the analysis shows that as the vibrational acceleration is increased, the pilot-wave field undergoes a series of Hopf bifurcations that ultimately lead to a chaotic wave field. The established relation between the mean pilot-wave field and the droplet statistics allows us to characterize the evolution of the emergent statistics with increased vibrational forcing from the evolution of the pilot-wave field. We thus develop a numerical framework with the same basic structure as quantum mechanics, specifically a wave theory that predicts particle statistics. △ Less

Submitted 13 October, 2022; originally announced October 2022.

Comments: 12 pages, 10 figures

Effcient magnetometer sensor array selection for signal reconstruction and brain source localization

Authors: Wan-** Yeo, Samu Taulu, J. Nathan Kutz

Abstract: Magnetoencephalography (MEG) is a noninvasive method for measuring magnetic flux signals caused by brain activity using sensor arrays located on or above the scalp. A common strategy for monitoring brain activity is to place sensors on a nearly uniform grid, or sensor array, around the head. By increasing the total number of sensors, higher spatial-frequency components of brain activity can be res… ▽ More Magnetoencephalography (MEG) is a noninvasive method for measuring magnetic flux signals caused by brain activity using sensor arrays located on or above the scalp. A common strategy for monitoring brain activity is to place sensors on a nearly uniform grid, or sensor array, around the head. By increasing the total number of sensors, higher spatial-frequency components of brain activity can be resolved as dictated by Nyquist sampling theory. Currently, there are few principled mathematical architectures for sensor placement aside from Nyquist considerations. However, global brain activity often exhibits low-dimensional patterns of spatio-temporal dynamics. The low-dimensional global patterns can be computed from the singular value decomposition and can be leveraged to select a small number of sensors optimized for reconstructing brain signals and localizing brain sources. Moreover, a smaller number of sensors which are systematically chosen can outperform the entire sensor array when considering noisy measurements. We propose a greedy selection algorithm based upon the QR decomposition that is computationally efficient to implement for MEG. We demonstrate the performance of the sensor selection algorithm for the tasks of signal reconstruction and localization. The performance is dependent upon source localization, with shallow sources easily identified and reconstructed, and deep sources more difficult to locate. Our findings suggest that principled methods for sensor selection can improve MEG capabilities and potentially add cost savings for monitoring brain-wide activity. △ Less

Submitted 22 May, 2022; originally announced May 2022.

The Experimental Multi-Arm Pendulum on a Cart: A Benchmark System for Chaos, Learning, and Control

Authors: Kadierdan Kaheman, Urban Fasel, Jason J. Bramburger, Benjamin Strom, J. Nathan Kutz, Steven L. Brunton

Abstract: The single, double, and triple pendulum has served as an illustrative experimental benchmark system for scientists to study dynamical behavior for more than four centuries. The pendulum system exhibits a wide range of interesting behaviors, from simple harmonic motion in the single pendulum to chaotic dynamics in multi-arm pendulums. Under forcing, even the single pendulum may exhibit chaos, provi… ▽ More The single, double, and triple pendulum has served as an illustrative experimental benchmark system for scientists to study dynamical behavior for more than four centuries. The pendulum system exhibits a wide range of interesting behaviors, from simple harmonic motion in the single pendulum to chaotic dynamics in multi-arm pendulums. Under forcing, even the single pendulum may exhibit chaos, providing a simple example of a damped-driven system. All multi-armed pendulums are characterized by the existence of index-one saddle points, which mediate the transport of trajectories in the system, providing a simple mechanical analog of various complex transport phenomena, from biolocomotion to transport within the solar system. Further, pendulum systems have long been used to design and test both linear and nonlinear control strategies, with the addition of more arms making the problem more challenging. In this work, we provide extensive designs for the construction and operation of a high-performance, multi-link pendulum on a cart system. Although many experimental setups have been built to study the behavior of pendulum systems, such an extensive documentation on the design, construction, and operation is missing from the literature. The resulting experimental system is highly flexible, enabling a wide range of benchmark problems in dynamical systems modeling, system identification and learning, and control. To promote reproducible research, we have made our entire system open-source, including 3D CAD drawings, basic tutorial code, and data. Moreover, we discuss the possibility of extending our system capability to be operated remotely to enable researchers all around the world to use it, thus increasing access. △ Less

Submitted 12 May, 2022; originally announced May 2022.

Comments: 72 pages, 39 figures, 9 tables

ACM Class: B.0; D.0; E.0

Transitions between peace and systemic war as bifurcations in a signed network dynamical system

Authors: Megan Morrison, J. Nathan Kutz, Michael Gabbay

Abstract: We investigate structural features and processes associated with the onset of systemic conflict using an approach which integrates complex systems theory with network modeling and analysis. We present a signed network model of cooperation and conflict dynamics in the context of international relations between states. The model evolves ties between nodes under the influence of a structural balance… ▽ More We investigate structural features and processes associated with the onset of systemic conflict using an approach which integrates complex systems theory with network modeling and analysis. We present a signed network model of cooperation and conflict dynamics in the context of international relations between states. The model evolves ties between nodes under the influence of a structural balance force and a dyad-specific force. Model simulations exhibit a sharp bifurcation from peace to systemic war as structural balance pressures increase, a bistable regime in which both peace and war stable equilibria exist, and a hysteretic reverse bifurcation from war to peace. We show how the analytical expression we derive for the peace-to-war bifurcation condition implies that polarized network structure increases susceptibility to systemic war. We develop a framework for identifying patterns of relationship perturbations that are most destabilizing and apply it to the network of European great powers before World War I. We also show that the model exhibits critical slowing down, in which perturbations to the peace equilibrium take longer to decay as the system draws closer to the bifurcation. We discuss how our results relate to international relations theories on the causes and catalysts of systemic war. △ Less

Submitted 8 March, 2022; originally announced March 2022.

MSC Class: 91D30; 37G99; 37N99; 91C20; 34H20 ACM Class: J.4

Dimensionally Consistent Learning with Buckingham Pi

Authors: Joseph Bakarji, Jared Callaham, Steven L. Brunton, J. Nathan Kutz

Abstract: In the absence of governing equations, dimensional analysis is a robust technique for extracting insights and finding symmetries in physical systems. Given measurement variables and parameters, the Buckingham Pi theorem provides a procedure for finding a set of dimensionless groups that spans the solution space, although this set is not unique. We propose an automated approach using the symmetric… ▽ More In the absence of governing equations, dimensional analysis is a robust technique for extracting insights and finding symmetries in physical systems. Given measurement variables and parameters, the Buckingham Pi theorem provides a procedure for finding a set of dimensionless groups that spans the solution space, although this set is not unique. We propose an automated approach using the symmetric and self-similar structure of available measurement data to discover the dimensionless groups that best collapse this data to a lower dimensional space according to an optimal fit. We develop three data-driven techniques that use the Buckingham Pi theorem as a constraint: (i) a constrained optimization problem with a non-parametric input-output fitting function, (ii) a deep learning algorithm (BuckiNet) that projects the input parameter space to a lower dimension in the first layer, and (iii) a technique based on sparse identification of nonlinear dynamics (SINDy) to discover dimensionless equations whose coefficients parameterize the dynamics. We explore the accuracy, robustness and computational complexity of these methods as applied to three example problems: a bead on a rotating hoop, a laminar boundary layer, and Rayleigh-Bénard convection. △ Less

Submitted 9 February, 2022; originally announced February 2022.

A new approach for determining optimal placement of PM2.5 air quality sensors: case study for the contiguous United States

Authors: Makoto M. Kelp, Samuel Lin, J. Nathan Kutz, Loretta J. Mickley

Abstract: Considerable financial resources are allocated for measuring ambient air pollution in the United States, yet the locations for these monitoring sites may not be optimized to capture the full extent of current pollution variability. Prior research on best sensor placement for monitoring fine particulate matter (PM2.5) pollution is scarce: most studies do not span areas larger than a medium-sized ci… ▽ More Considerable financial resources are allocated for measuring ambient air pollution in the United States, yet the locations for these monitoring sites may not be optimized to capture the full extent of current pollution variability. Prior research on best sensor placement for monitoring fine particulate matter (PM2.5) pollution is scarce: most studies do not span areas larger than a medium-sized city or examine timescales longer than one week. Here we present a pilot study using multiresolution modal decomposition (mrDMD) to identify the optimal placement of PM2.5 sensors from 2000-2016 over the contiguous United States. This novel approach incorporates the variation of PM2.5 on timescales ranging from one day to over a decade to capture air pollution variability. We find that the mrDMD algorithm identifies high-priority sensor locations in the western United States, but a significantly lower density of sensors than expected along the eastern coast, where a large number of EPA PM2.5 monitors currently reside. Specifically, 53% of mrDMD optimized sensor locations are west of the 100th meridian, compared to only 32% in the current EPA network. The mrDMD sensor locations can capture PM2.5 from wildfires and high pollution events, with particularly high skill in the West. These results suggest significant gaps in the current EPA monitoring network in the San Joaquin Valley in California, northern California, and in the Pacific Northwest (Idaho, and Eastern Washington and Oregon). Our framework diagnoses where to place air quality sensors so that they can best monitor smoke from wildfires. Our framework may also be applied to urban areas for equitable placement of PM2.5 monitors. △ Less

Submitted 16 February, 2022; v1 submitted 4 January, 2022; originally announced January 2022.

Comments: 18 pages, 3 figures, now published at Environmental Research Letters: https://doi.org/10.1088/1748-9326/ac548f

Physics-informed dynamic mode decomposition (piDMD)

Authors: Peter J. Baddoo, Benjamin Herrmann, Beverley J. McKeon, J. Nathan Kutz, Steven L. Brunton

Abstract: In this work, we demonstrate how physical principles -- such as symmetries, invariances, and conservation laws -- can be integrated into the dynamic mode decomposition (DMD). DMD is a widely-used data analysis technique that extracts low-rank modal structures and dynamics from high-dimensional measurements. However, DMD frequently produces models that are sensitive to noise, fail to generalize out… ▽ More In this work, we demonstrate how physical principles -- such as symmetries, invariances, and conservation laws -- can be integrated into the dynamic mode decomposition (DMD). DMD is a widely-used data analysis technique that extracts low-rank modal structures and dynamics from high-dimensional measurements. However, DMD frequently produces models that are sensitive to noise, fail to generalize outside the training data, and violate basic physical laws. Our physics-informed DMD (piDMD) optimization, which may be formulated as a Procrustes problem, restricts the family of admissible models to a matrix manifold that respects the physical structure of the system. We focus on five fundamental physical principles -- conservation, self-adjointness, localization, causality, and shift-invariance -- and derive several closed-form solutions and efficient algorithms for the corresponding piDMD optimizations. With fewer degrees of freedom, piDMD models are less prone to overfitting, require less training data, and are often less computationally expensive to build than standard DMD models. We demonstrate piDMD on a range of challenging problems in the physical sciences, including energy-preserving fluid flow, travelling-wave systems, the Schrödinger equation, solute advection-diffusion, a system with causal dynamics, and three-dimensional transitional channel flow. In each case, piDMD significantly outperforms standard DMD in metrics such as spectral identification, state prediction, and estimation of optimal forcings and responses. △ Less

Submitted 8 December, 2021; originally announced December 2021.

Comments: 36 pages, 9 figures

PySINDy: A comprehensive Python package for robust sparse system identification

Authors: Alan A. Kaptanoglu, Brian M. de Silva, Urban Fasel, Kadierdan Kaheman, Andy J. Goldschmidt, Jared L. Callaham, Charles B. Delahunt, Zachary G. Nicolaou, Kathleen Champion, Jean-Christophe Loiseau, J. Nathan Kutz, Steven L. Brunton

Abstract: Automated data-driven modeling, the process of directly discovering the governing equations of a system from data, is increasingly being used across the scientific community. PySINDy is a Python package that provides tools for applying the sparse identification of nonlinear dynamics (SINDy) approach to data-driven model discovery. In this major update to PySINDy, we implement several advanced feat… ▽ More Automated data-driven modeling, the process of directly discovering the governing equations of a system from data, is increasingly being used across the scientific community. PySINDy is a Python package that provides tools for applying the sparse identification of nonlinear dynamics (SINDy) approach to data-driven model discovery. In this major update to PySINDy, we implement several advanced features that enable the discovery of more general differential equations from noisy and limited data. The library of candidate terms is extended for the identification of actuated systems, partial differential equations (PDEs), and implicit differential equations. Robust formulations, including the integral form of SINDy and ensembling techniques, are also implemented to improve performance for real-world data. Finally, we provide a range of new optimization algorithms, including several sparse regression techniques and algorithms to enforce and promote inequality constraints and stability. Together, these updates enable entirely new SINDy model discovery capabilities that have not been reported in the literature, such as constrained PDE identification and ensembling with different sparse regression optimizers. △ Less

Submitted 25 January, 2022; v1 submitted 12 November, 2021; originally announced November 2021.

Dynamic Mode Decomposition for Aero-Optic Wavefront Characterization

Authors: Shervin Sahba, Diya Sashidhar, Christopher C. Wilcox, Austin McDaniel, Steven L. Brunton, J. Nathan Kutz

Abstract: Aero-optical beam control relies on the development of low-latency forecasting techniques to quickly predict wavefronts aberrated by the Turbulent Boundary Layer (TBL) around an airborne optical system, and its study applies to a multi-domain need from astronomy to microscopy for high-fidelity laser propagation. We leverage the forecasting capabilities of the Dynamic Mode Decomposition (DMD) -- an… ▽ More Aero-optical beam control relies on the development of low-latency forecasting techniques to quickly predict wavefronts aberrated by the Turbulent Boundary Layer (TBL) around an airborne optical system, and its study applies to a multi-domain need from astronomy to microscopy for high-fidelity laser propagation. We leverage the forecasting capabilities of the Dynamic Mode Decomposition (DMD) -- an equation-free, data-driven method for identifying coherent flow structures and their associated spatiotemporal dynamics -- in order to estimate future state wavefront phase aberrations to feed into an adaptive optic (AO) control loop. We specifically leverage the optimized DMD (opt-DMD) algorithm on a subset of the Airborne Aero-Optics Laboratory Transonic (AAOL-T) experimental dataset, characterizing aberrated wavefront dynamics for 23 beam propagation directions via the spatiotemporal decomposition underlying DMD. Critically, we show that opt-DMD produces an optimally de-biased eigenvalue spectrum with imaginary eigenvalues, allowing for arbitrarily long forecasting to produce a robust future-state prediction, while exact DMD loses structural information due to modal decay rates. △ Less

Submitted 13 January, 2022; v1 submitted 28 September, 2021; originally announced October 2021.

Comments: 15 pages, 8 figures, the two first-authors contributed equally to this work

Data-driven Modeling of Two-Dimensional Detonation Wave Fronts

Authors: Ariana Mendible, Weston Lowrie, Steven L. Brunton, J. Nathan Kutz

Abstract: Historical experimental testing of high-altitude nuclear explosions (HANEs) are known to cause severe and detrimental effects to radio frequency signals and communications infrastructure. In order to study and predict the impact of HANEs, tractable computational approaches are required to model the complex physical processes involved in the detonation wave physics. Modern reduced-order models (ROM… ▽ More Historical experimental testing of high-altitude nuclear explosions (HANEs) are known to cause severe and detrimental effects to radio frequency signals and communications infrastructure. In order to study and predict the impact of HANEs, tractable computational approaches are required to model the complex physical processes involved in the detonation wave physics. Modern reduced-order models (ROMs) can enable long-time and many-parameter simulations with minimal computational cost. However, translational and scale invariances inherent to this type of wave propagation problem are known to limit traditional ROM approaches. Specifically, dimensionality reduction methods are typically ineffective in producing low-rank models when invariances are present in the data. In this work, an unsupervised machine learning method is used to discover coordinate systems that make such invariances amenable to traditional dimensionality reduction methods. The method, which has previously been demonstrated on one-dimensional translations, is extended to higher dimensions and additional invariances. A surrogate HANE system, i.e. a HANE-ROM, with one detonation wave is captured well at extremely low-rank. Two detonation-waves are also considered with various amounts of interaction between the waves, with improvements to low-rank models for multiple wave quantities with limited interaction. △ Less

Submitted 30 June, 2021; originally announced June 2021.

Comments: 22 pages, 13 figures

DeepGreen: Deep Learning of Green's Functions for Nonlinear Boundary Value Problems

Authors: Craig R. Gin, Daniel E. Shea, Steven L. Brunton, J. Nathan Kutz

Abstract: Boundary value problems (BVPs) play a central role in the mathematical analysis of constrained physical systems subjected to external forces. Consequently, BVPs frequently emerge in nearly every engineering discipline and span problem domains including fluid mechanics, electromagnetics, quantum mechanics, and elasticity. The fundamental solution, or Green's function, is a leading method for solvin… ▽ More Boundary value problems (BVPs) play a central role in the mathematical analysis of constrained physical systems subjected to external forces. Consequently, BVPs frequently emerge in nearly every engineering discipline and span problem domains including fluid mechanics, electromagnetics, quantum mechanics, and elasticity. The fundamental solution, or Green's function, is a leading method for solving linear BVPs that enables facile computation of new solutions to systems under any external forcing. However, fundamental Green's function solutions for nonlinear BVPs are not feasible since linear superposition no longer holds. In this work, we propose a flexible deep learning approach to solve nonlinear BVPs using a dual-autoencoder architecture. The autoencoders discover an invertible coordinate transform that linearizes the nonlinear BVP and identifies both a linear operator $L$ and Green's function $G$ which can be used to solve new nonlinear BVPs. We find that the method succeeds on a variety of nonlinear systems including nonlinear Helmholtz and Sturm--Liouville problems, nonlinear elasticity, and a 2D nonlinear Poisson equation. The method merges the strengths of the universal approximation capabilities of deep learning with the physics knowledge of Green's functions to yield a flexible tool for identifying fundamental solutions to a variety of nonlinear systems. △ Less

Submitted 31 December, 2020; originally announced January 2021.

Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems

Authors: Daniel Dylewsky, David Barajas-Solano, Tong Ma, Alexandre M. Tartakovsky, J. Nathan Kutz

Abstract: Time series forecasting remains a central challenge problem in almost all scientific disciplines. We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system using Dynamic Mode Decomposition (DMD) in time delay coordinates. Central to this approach is the insight that grid load, like many observables on complex real-world systems, has an "almost-pe… ▽ More Time series forecasting remains a central challenge problem in almost all scientific disciplines. We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system using Dynamic Mode Decomposition (DMD) in time delay coordinates. Central to this approach is the insight that grid load, like many observables on complex real-world systems, has an "almost-periodic" character, i.e., a continuous Fourier spectrum punctuated by dominant peaks, which capture regular (e.g., daily or weekly) recurrences in the dynamics. The forecasting method presented takes advantage of this property by (i) regressing to a deterministic linear model whose eigenspectrum maps onto those peaks, and (ii) simultaneously learning a stochastic Gaussian process regression (GPR) process to actuate this system. Our forecasting algorithm is compared against state-of-the-art forecasting techniques not using additional explanatory variables and is shown to produce superior performance. Moreover, its use of linear intrinsic dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid. Load forecasting is an essential challenge in power systems engineering, with major implications for real-time control, pricing, maintenance, and security decisions. △ Less

Submitted 9 July, 2021; v1 submitted 8 October, 2020; originally announced October 2020.

Hierarchical Deep Learning of Multiscale Differential Equation Time-Steppers

Authors: Yuying Liu, J. Nathan Kutz, Steven L. Brunton

Abstract: Nonlinear differential equations rarely admit closed-form solutions, thus requiring numerical time-step** algorithms to approximate solutions. Further, many systems characterized by multiscale physics exhibit dynamics over a vast range of timescales, making numerical integration computationally expensive due to numerical stiffness. In this work, we develop a hierarchy of deep neural network time… ▽ More Nonlinear differential equations rarely admit closed-form solutions, thus requiring numerical time-step** algorithms to approximate solutions. Further, many systems characterized by multiscale physics exhibit dynamics over a vast range of timescales, making numerical integration computationally expensive due to numerical stiffness. In this work, we develop a hierarchy of deep neural network time-steppers to approximate the flow map of the dynamical system over a disparate range of time-scales. The resulting model is purely data-driven and leverages features of the multiscale dynamics, enabling numerical integration and forecasting that is both accurate and highly efficient. Moreover, similar ideas can be used to couple neural network-based models with classical numerical time-steppers. Our multiscale hierarchical time-step** scheme provides important advantages over current time-step** algorithms, including (i) circumventing numerical stiffness due to disparate time-scales, (ii) improved accuracy in comparison with leading neural-network architectures, (iii) efficiency in long-time simulation/forecasting due to explicit training of slow time-scale dynamics, and (iv) a flexible framework that is parallelizable and may be integrated with standard numerical time-step** algorithms. The method is demonstrated on a wide range of nonlinear dynamical systems, including the Van der Pol oscillator, the Lorenz system, the Kuramoto-Sivashinsky equation, and fluid flow pass a cylinder; audio and video signals are also explored. On the sequence generation examples, we benchmark our algorithm against state-of-the-art methods, such as LSTM, reservoir computing, and clockwork RNN. Despite the structural simplicity of our method, it outperforms competing methods on numerical integration. △ Less

Submitted 22 August, 2020; originally announced August 2020.

Comments: 22 pages, 10 figures

Data-driven Modeling of Rotating Detonation Waves

Authors: Ariana Mendible, James Koch, Henning Lange, Steven L. Brunton, J. Nathan Kutz

Abstract: The direct monitoring of a rotating detonation engine (RDE) combustion chamber has enabled the observation of combustion front dynamics that are composed of a number of co- and/or counter-rotating coherent traveling shock waves whose nonlinear mode-locking behavior exhibit bifurcations and instabilities which are not well understood. Computational fluid dynamics simulations are ubiquitous in chara… ▽ More The direct monitoring of a rotating detonation engine (RDE) combustion chamber has enabled the observation of combustion front dynamics that are composed of a number of co- and/or counter-rotating coherent traveling shock waves whose nonlinear mode-locking behavior exhibit bifurcations and instabilities which are not well understood. Computational fluid dynamics simulations are ubiquitous in characterizing the dynamics of RDE's reactive, compressible flow. Such simulations are prohibitively expensive when considering multiple engine geometries, different operating conditions, and the long-time dynamics of the mode-locking interactions. Reduced-order models (ROMs) provide a critically enabling simulation framework because they exploit low-rank structure in the data to minimize computational cost and allow for rapid parameterized studies and long-time simulations. However, ROMs are inherently limited by translational invariances manifest by the combustion waves present in RDEs. In this work, we leverage machine learning algorithms to discover moving coordinate frames into which the data is shifted, thus overcoming limitations imposed by the underlying translational invariance of the RDE and allowing for the application of traditional dimensionality reduction techniques. We explore a diverse suite of data-driven ROM strategies for characterizing the complex shock wave dynamics and interactions in the RDE. Specifically, we employ the dynamic mode decomposition and a deep Koopman embedding to give new modeling insights and understanding of combustion wave interactions in RDEs. △ Less

Submitted 12 August, 2020; originally announced August 2020.

Comments: 17 pages, 12 figures

Journal ref: Phys. Rev. Fluids 6, 050507 (2021)

Modeling Thermodynamic Trends of Rotating Detonation Engines

Authors: James Koch, J. Nathan Kutz

Abstract: The formation of a number of co- and counter-rotating coherent combustion wave fronts is the hallmark feature of the Rotating Detonation Engine (RDE). The engineering implications of wave topology are not well understood nor quantified, especially with respect to parametric changes in combustor geometry, propellant chemistry, and injection and mixing schemes. In this article, a modeling framework… ▽ More The formation of a number of co- and counter-rotating coherent combustion wave fronts is the hallmark feature of the Rotating Detonation Engine (RDE). The engineering implications of wave topology are not well understood nor quantified, especially with respect to parametric changes in combustor geometry, propellant chemistry, and injection and mixing schemes. In this article, a modeling framework that relates the time and spacial scales of the RDE to engineering performance metrics is developed and presented. The model is built under assumptions of backpressure-insensitivity and nominally choked gaseous propellant injection. The Euler equations of inviscid, compressible fluid flow in one dimension are adapted to model the combustion wave dynamics along the circumference of an annular-type rotating detonation engine. These adaptations provide the necessary mass and energy input and output channels to shape the traveling wave fronts and decaying tails. The associated unit processes of injection, mixing, combustion, and exhaust are all assigned representative time scales necessary for successful wave propagation. We find that the separation, or lack of, these time scales are responsible for the behavior of the system, including wave co- and counter-propagation and bifurcations between these regimes and wave counts. Furthermore, as there is no imposition of wave topology, the model output is used to estimate the net available mechanical work output and thermodynamic efficiency from the closed trajectories through pressure-volume and temperature-entropy spaces. These metrics are investigated with respect to variation in the characteristic scales for the RDE unit physical processes. △ Less

Submitted 29 July, 2020; originally announced July 2020.

Deep reinforcement learning for optical systems: A case study of mode-locked lasers

Authors: Chang Sun, Eurika Kaiser, Steven L. Brunton, J. Nathan Kutz

Abstract: We demonstrate that deep reinforcement learning (deep RL) provides a highly effective strategy for the control and self-tuning of optical systems. Deep RL integrates the two leading machine learning architectures of deep neural networks and reinforcement learning to produce robust and stable learning for control. Deep RL is ideally suited for optical systems as the tuning and control relies on int… ▽ More We demonstrate that deep reinforcement learning (deep RL) provides a highly effective strategy for the control and self-tuning of optical systems. Deep RL integrates the two leading machine learning architectures of deep neural networks and reinforcement learning to produce robust and stable learning for control. Deep RL is ideally suited for optical systems as the tuning and control relies on interactions with its environment with a goal-oriented objective to achieve optimal immediate or delayed rewards. This allows the optical system to recognize bi-stable structures and navigate, via trajectory planning, to optimally performing solutions, the first such algorithm demonstrated to do so in optical systems. We specifically demonstrate the deep RL architecture on a mode-locked laser, where robust self-tuning and control can be established through access of the deep RL agent to its waveplates and polarizers. We further integrate transfer learning to help the deep RL agent rapidly learn new parameter regimes and generalize its control authority. Additionally, the deep RL learning can be easily integrated with other control paradigms to provide a broad framework to control any optical system. △ Less

Submitted 9 June, 2020; originally announced June 2020.

Principal component trajectories for modeling spectrally-continuous dynamics as forced linear systems

Authors: Daniel Dylewsky, Eurika Kaiser, Steven L. Brunton, J. Nathan Kutz

Abstract: Delay embeddings of time series data have emerged as a promising coordinate basis for data-driven estimation of the Koopman operator, which seeks a linear representation for observed nonlinear dynamics. Recent work has demonstrated the efficacy of Dynamic Mode Decomposition (DMD) for obtaining finite-dimensional Koopman approximations in delay coordinates. In this paper we demonstrate how nonlinea… ▽ More Delay embeddings of time series data have emerged as a promising coordinate basis for data-driven estimation of the Koopman operator, which seeks a linear representation for observed nonlinear dynamics. Recent work has demonstrated the efficacy of Dynamic Mode Decomposition (DMD) for obtaining finite-dimensional Koopman approximations in delay coordinates. In this paper we demonstrate how nonlinear dynamics with sparse Fourier spectra can be (i) represented by a superposition of principal component trajectories (PCT) and (ii) modeled by DMD in this coordinate space. For continuous or mixed (discrete and continuous) spectra, DMD can be augmented with an external forcing term. We present a method for learning linear control models in delay coordinates while simultaneously discovering the corresponding exogeneous forcing signal in a fully unsupervised manner. This extends the existing DMD with control (DMDc) algorithm to cases where a control signal is not known a priori. We provide examples to validate the learned forcing against a known ground truth and illustrate their statistical similarity. Finally we offer a demonstration of this method applied to real-world power grid load data to show its utility for diagnostics and interpretation on systems in which somewhat periodic behavior is strongly forced by unknown and unmeasurable environmental variables. △ Less

Submitted 12 January, 2022; v1 submitted 28 May, 2020; originally announced May 2020.

Spatio-Temporal Mode-locking in Quadratic Nonlinear Media

Authors: Mahmut Bağcı, J. Nathan Kutz

Abstract: A new theoretical model is developed to characterize spatio-temporal mode-locking (ML) in quadratic nonlinear media. The model is based on the two-dimensional nonlinear Schrödinger equation with coupling to a mean term (NLSM) and constructed as an extension of the master mode-locking model. It is numerically demonstrated that there exists steady state soliton solutions of the ML-NLSM mode-locking… ▽ More A new theoretical model is developed to characterize spatio-temporal mode-locking (ML) in quadratic nonlinear media. The model is based on the two-dimensional nonlinear Schrödinger equation with coupling to a mean term (NLSM) and constructed as an extension of the master mode-locking model. It is numerically demonstrated that there exists steady state soliton solutions of the ML-NLSM mode-locking model that are astigmatic in nature. A full stability analysis and bifurcation study is performed for the ML-NLSM model and it is manifest that spatio-temporal mode-locking of the astigmatic steady-state solutions is possible in quadratic nonlinear media. △ Less

Submitted 21 May, 2020; originally announced May 2020.

Comments: 9 pages, 7 Figures

Journal ref: Phys. Rev. E 102, 022205 (2020)

PySINDy: A Python package for the Sparse Identification of Nonlinear Dynamics from Data

Authors: Brian M. de Silva, Kathleen Champion, Markus Quade, Jean-Christophe Loiseau, J. Nathan Kutz, Steven L. Brunton

Abstract: PySINDy is a Python package for the discovery of governing dynamical systems models from data. In particular, PySINDy provides tools for applying the sparse identification of nonlinear dynamics (SINDy) (Brunton et al. 2016) approach to model discovery. In this work we provide a brief description of the mathematical underpinnings of SINDy, an overview and demonstration of the features implemented i… ▽ More PySINDy is a Python package for the discovery of governing dynamical systems models from data. In particular, PySINDy provides tools for applying the sparse identification of nonlinear dynamics (SINDy) (Brunton et al. 2016) approach to model discovery. In this work we provide a brief description of the mathematical underpinnings of SINDy, an overview and demonstration of the features implemented in PySINDy (with code examples), practical advice for users, and a list of potential extensions to PySINDy. Software is available at https://github.com/dynamicslab/pysindy. △ Less

Submitted 17 April, 2020; originally announced April 2020.

SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics

Authors: Kadierdan Kaheman, J. Nathan Kutz, Steven L. Brunton

Abstract: Accurately modeling the nonlinear dynamics of a system from measurement data is a challenging yet vital topic. The sparse identification of nonlinear dynamics (SINDy) algorithm is one approach to discover dynamical systems models from data. Although extensions have been developed to identify implicit dynamics, or dynamics described by rational functions, these extensions are extremely sensitive to… ▽ More Accurately modeling the nonlinear dynamics of a system from measurement data is a challenging yet vital topic. The sparse identification of nonlinear dynamics (SINDy) algorithm is one approach to discover dynamical systems models from data. Although extensions have been developed to identify implicit dynamics, or dynamics described by rational functions, these extensions are extremely sensitive to noise. In this work, we develop SINDy-PI (parallel, implicit), a robust variant of the SINDy algorithm to identify implicit dynamics and rational nonlinearities. The SINDy-PI framework includes multiple optimization algorithms and a principled approach to model selection. We demonstrate the ability of this algorithm to learn implicit ordinary and partial differential equations and conservation laws from limited and noisy data. In particular, we show that the proposed approach is several orders of magnitude more noise robust than previous approaches, and may be used to identify a class of complex ODE and PDE dynamics that were previously unattainable with SINDy, including for the double pendulum dynamics and the Belousov Zhabotinsky (BZ) reaction. △ Less

Submitted 29 September, 2020; v1 submitted 5 April, 2020; originally announced April 2020.

Comments: 25 pages, 9 figures, 5 tables

MSC Class: 93B30

Multi-scale Physics of Rotating Detonation Engines: Autosolitons and Modulational Instabilities

Authors: James Koch, Mitsuru Kurosaka, Carl Knowlen, J. Nathan Kutz

Abstract: We develop a theoretical framework that predicts and fully characterizes the diverse experimental observations of the nonlinear, combustion wave propagation in a rotating detonation engine (RDE), including the nucleation and formation of combustion pulses, the soliton-like interactions between these combustion fronts, and the fundamental, underlying Hopf bifurcation to time-periodic modulation of… ▽ More We develop a theoretical framework that predicts and fully characterizes the diverse experimental observations of the nonlinear, combustion wave propagation in a rotating detonation engine (RDE), including the nucleation and formation of combustion pulses, the soliton-like interactions between these combustion fronts, and the fundamental, underlying Hopf bifurcation to time-periodic modulation of the waves. In this framework, the mode-locked structures are classified as autosolitons, or stably-propagating nonlinear waves where the local physics of nonlinearity, dispersion, gain, and dissipation exactly balance. We find that the global dominant balance physics in the RDE combustion chamber are dissipative and multi-scale in nature, with local fast scale (nano- to microseconds) combustion balances generating the fundamental mode-locked autosoliton state, while slow scale (milliseconds) gain-loss balances determine the instabilities and structure of the total number of autosolitons. In this manner, the global multi-scale balance physics give rise to the stable structures - not exclusively the frontal dynamics prescribed by classical detonation theory. Experimental observations and numerical models of the RDE combustion chamber are in strong qualitative agreement with no parameter tuning. Moreover, numerical continuation (computational bifurcation tracking) of the RDE analog system establishes that a Hopf bifurcation of the steadily propagating pulse train leads to the fundamental instability of the RDE, or time-periodic modulation of the waves. Along branches of Hopf orbits in parameter space exist a continuum of wave-pair interactions that exhibit solitonic interactions of varying strength. △ Less

Submitted 14 March, 2020; originally announced March 2020.

Comments: 18 pages, 13 figures

Journal ref: Phys. Rev. E 104, 024210 (2021)

Learning dominant physical processes with data-driven balance models

Authors: Jared L. Callaham, James V. Koch, Bingni W. Brunton, J. Nathan Kutz, Steven L. Brunton

Abstract: Throughout the history of science, physics-based modeling has relied on judiciously approximating observed dynamics as a balance between a few dominant processes. However, this traditional approach is mathematically cumbersome and only applies in asymptotic regimes where there is a strict separation of scales in the physics. Here, we automate and generalize this approach to non-asymptotic regimes… ▽ More Throughout the history of science, physics-based modeling has relied on judiciously approximating observed dynamics as a balance between a few dominant processes. However, this traditional approach is mathematically cumbersome and only applies in asymptotic regimes where there is a strict separation of scales in the physics. Here, we automate and generalize this approach to non-asymptotic regimes by introducing the idea of an equation space, in which different local balances appear as distinct subspace clusters. Unsupervised learning can then automatically identify regions where groups of terms may be neglected. We show that our data-driven balance models successfully delineate dominant balance physics in a much richer class of systems. In particular, this approach uncovers key mechanistic models in turbulence, combustion, nonlinear optics, geophysical fluids, and neuroscience. △ Less

Submitted 19 January, 2021; v1 submitted 27 January, 2020; originally announced January 2020.

Comments: 30 pages, 13 figures

Journal ref: Nature Communications 12, 1016 (2021)

Nonlinear control in the nematode C. elegans

Authors: Megan Morrison, Charles Fieseler, J. Nathan Kutz

Abstract: Recent whole-brain calcium imaging recordings of the nematode C. elegans have demonstrated that neural activity is dominated by dynamics on a low-dimensional manifold that can be clustered according to behavioral states. Despite progress in modeling the dynamics with linear or locally linear models, it remains unclear how a single network of neurons can produce the observed features. In particular… ▽ More Recent whole-brain calcium imaging recordings of the nematode C. elegans have demonstrated that neural activity is dominated by dynamics on a low-dimensional manifold that can be clustered according to behavioral states. Despite progress in modeling the dynamics with linear or locally linear models, it remains unclear how a single network of neurons can produce the observed features. In particular, there are multiple clusters, or fixed points, observed in the data which cannot be characterized by a single linear model. We propose a nonlinear control model which is global and parameterized by only four free parameters that match the features displayed by the low-dimensional C. elegans neural activity. In addition to reproducing the average probability distribution of the data, long and short time-scale changes in transition statistics can be characterized via changes in a single parameter. Some of these macro-scale transitions have experimental correlates to single neuro-modulators that seem to act as biological controls, allowing this model to generate testable hypotheses about the effect of these neuro-modulators on the global dynamics. The theory provides an elegant characterization of the neuron population dynamics in C. elegans. Moreover, the mathematical structure of the nonlinear control framework provides a paradigm that can be generalized to more complex systems with an arbitrary number of behavioral states. △ Less

Submitted 3 February, 2020; v1 submitted 22 January, 2020; originally announced January 2020.

MSC Class: 92B05 ACM Class: J.3

Deep Learning Models for Global Coordinate Transformations that Linearize PDEs

Authors: Craig Gin, Bethany Lusch, Steven L. Brunton, J. Nathan Kutz

Abstract: We develop a deep autoencoder architecture that can be used to find a coordinate transformation which turns a nonlinear PDE into a linear PDE. Our architecture is motivated by the linearizing transformations provided by the Cole-Hopf transform for Burgers equation and the inverse scattering transform for completely integrable PDEs. By leveraging a residual network architecture, a near-identity tra… ▽ More We develop a deep autoencoder architecture that can be used to find a coordinate transformation which turns a nonlinear PDE into a linear PDE. Our architecture is motivated by the linearizing transformations provided by the Cole-Hopf transform for Burgers equation and the inverse scattering transform for completely integrable PDEs. By leveraging a residual network architecture, a near-identity transformation can be exploited to encode intrinsic coordinates in which the dynamics are linear. The resulting dynamics are given by a Koopman operator matrix $\mathbf{K}$. The decoder allows us to transform back to the original coordinates as well. Multiple time step prediction can be performed by repeated multiplication by the matrix $\mathbf{K}$ in the intrinsic coordinates. We demonstrate our method on a number of examples, including the heat equation and Burgers equation, as well as the substantially more challenging Kuramoto-Sivashinsky equation, showing that our method provides a robust architecture for discovering interpretable, linearizing transforms for nonlinear PDEs. △ Less

Submitted 6 November, 2019; originally announced November 2019.

Comments: 23 pages, 18 figures

MSC Class: 35A22; 35A35; 37M99; 65P99; 68T99

Dimensionality Reduction and Reduced Order Modeling for Traveling Wave Physics

Authors: Ariana Mendible, Steven L. Brunton, Aleksandr Y. Aravkin, Wes Lowrie, J. Nathan Kutz

Abstract: We develop an unsupervised machine learning algorithm for the automated discovery and identification of traveling waves in spatio-temporal systems governed by partial differential equations (PDEs). Our method uses sparse regression and subspace clustering to robustly identify translational invariances that can be leveraged to build improved reduced order models (ROMs). Invariances, whether transla… ▽ More We develop an unsupervised machine learning algorithm for the automated discovery and identification of traveling waves in spatio-temporal systems governed by partial differential equations (PDEs). Our method uses sparse regression and subspace clustering to robustly identify translational invariances that can be leveraged to build improved reduced order models (ROMs). Invariances, whether translational or rotational, are well known to compromise the ability of ROMs to produce accurate and/or low-rank representations of the spatio-temporal dynamics. However, by discovering translations in a principled way, data can be shifted into a coordinate systems where quality, low-dimensional ROMs can be constructed. This approach can be used on either numerical or experimental data with or without knowledge of the governing equations. We demonstrate our method on a variety of PDEs of increasing difficulty, taken from the field of fluid dynamics, showing the efficacy and robustness of the proposed approach. △ Less

Submitted 18 May, 2020; v1 submitted 1 November, 2019; originally announced November 2019.

Comments: 14 pages, 8 figures

Mode-Locked Rotating Detonation Waves: Experiments and a Model Equation

Authors: James Koch, Mitsuru Kurosaka, Carl Knowlen, J. Nathan Kutz

Abstract: Direct observation of a Rotating Detonation Engine combustion chamber has enabled the extraction of the kinematics of its detonation waves. These records exhibit a rich set of instabilities and bifurcations arising from the interaction of coherent wave fronts and global gain dynamics. We develop a model of the observed dynamics by recasting the Majda detonation analog as an autowave. The solution… ▽ More Direct observation of a Rotating Detonation Engine combustion chamber has enabled the extraction of the kinematics of its detonation waves. These records exhibit a rich set of instabilities and bifurcations arising from the interaction of coherent wave fronts and global gain dynamics. We develop a model of the observed dynamics by recasting the Majda detonation analog as an autowave. The solution fronts become attractors of the engine; i.e., mode-locked rotating detonation waves. We find that detonative energy release competes with dissipation and gain recovery to produce the observed dynamics and a bifurcation structure common to driven-dissipative systems, such as mode-locked lasers. △ Less

Submitted 4 November, 2019; v1 submitted 8 August, 2019; originally announced August 2019.

Comments: 10 pages, 11 figures

Journal ref: Phys. Rev. E 101, 013106 (2020)

Frequency comb generation at 800nm in waveguide array quantum well diode lasers

Authors: Chang Sun, Mark Dong, Niall M. Mangan, Herbert G. Winful, Steven T. Cundiff, J. Nathan Kutz

Abstract: A traveling wave model for a semiconductor diode laser based on quantum wells is presented as well as a comprehensive theoretical model of the lasing dynamics produced by the intensity discrimination of the nonlinear mode-coupling in a waveguide array. By leveraging a recently developed model for the detailed semiconductor gain dynamics, the temporal sha** effects of the nonlinear mode-coupling… ▽ More A traveling wave model for a semiconductor diode laser based on quantum wells is presented as well as a comprehensive theoretical model of the lasing dynamics produced by the intensity discrimination of the nonlinear mode-coupling in a waveguide array. By leveraging a recently developed model for the detailed semiconductor gain dynamics, the temporal sha** effects of the nonlinear mode-coupling induced by the waveguide arrays can be characterized. Specifically, the enhanced nonlinear pulse sha** provided by the waveguides are capable of generating stable frequency combs wavelength of 800 nm in a GaAs device, a parameter regime not feasible for stable combline generation using a single waveguide. Extensive numerical simulations showed that stable waveform generation could be achieved and optimized by an appropriate choice of the linear waveguide coupling coefficient, quantum well depth, and the input currents to the first and second waveguides. The model provides a first demonstration that a compact, efficient and robust on-chip comb source can be produced in GaAs. △ Less

Submitted 25 June, 2019; originally announced June 2019.

Comments: 9 pages, 10 figures. arXiv admin note: substantial text overlap with arXiv:1707.01582, arXiv:1906.11097

Stable Numerical Schemes for Nonlinear Dispersive Equations with Counter-Propagation and Gain Dynamics

Authors: Chang Sun, Niall Mangan, Mark Dong, Herbert G. Winful, Steven T. Cundiff, J. Nathan Kutz

Abstract: We develop a stable and efficient numerical scheme for modeling the optical field evolution in a nonlinear dispersive cavity with counter propagating waves and complex, semiconductor physics gain dynamics that are expensive to evaluate. Our stability analysis is characterized by a von-Neumann analysis which shows that many standard numerical schemes are unstable due to competing physical effects i… ▽ More We develop a stable and efficient numerical scheme for modeling the optical field evolution in a nonlinear dispersive cavity with counter propagating waves and complex, semiconductor physics gain dynamics that are expensive to evaluate. Our stability analysis is characterized by a von-Neumann analysis which shows that many standard numerical schemes are unstable due to competing physical effects in the propagation equations. We show that the combination of a predictor-corrector scheme with an operator-splitting not only results in a stable scheme, but provides a highly efficient, single-stage evaluation of the gain dynamics. Given that the gain dynamics is the rate-limiting step of the algorithm, our method circumvents the numerical instability induced by the other cavity physics when evaluating the gain in an efficient manner. We demonstrate the stability and efficiency of the algorithm on a diode laser model which includes three waveguides and semiconductor gain dynamics. The laser is able to produce a repeating temporal waveform and stable optical comblines, thus demonstrating that frequency combs generation may be possible in chip scale, diode lasers. △ Less

Submitted 25 June, 2019; originally announced June 2019.

Comments: 11 pages, 5 figures. arXiv admin note: text overlap with arXiv:1707.01582

A unified sparse optimization framework to learn parsimonious physics-informed models from data

Authors: Kathleen Champion, Peng Zheng, Aleksandr Y. Aravkin, Steven L. Brunton, J. Nathan Kutz

Abstract: Machine learning (ML) is redefining what is possible in data-intensive fields of science and engineering. However, applying ML to problems in the physical sciences comes with a unique set of challenges: scientists want physically interpretable models that can (i) generalize to predict previously unobserved behaviors, (ii) provide effective forecasting predictions (extrapolation), and (iii) be cert… ▽ More Machine learning (ML) is redefining what is possible in data-intensive fields of science and engineering. However, applying ML to problems in the physical sciences comes with a unique set of challenges: scientists want physically interpretable models that can (i) generalize to predict previously unobserved behaviors, (ii) provide effective forecasting predictions (extrapolation), and (iii) be certifiable. Autonomous systems will necessarily interact with changing and uncertain environments, motivating the need for models that can accurately extrapolate based on physical principles (e.g. Newton's universal second law for classical mechanics, $F=ma$). Standard ML approaches have shown impressive performance for predicting dynamics in an interpolatory regime, but the resulting models often lack interpretability and fail to generalize. We introduce a unified sparse optimization framework that learns governing dynamical systems models from data, selecting relevant terms in the dynamics from a library of possible functions. The resulting models are parsimonious, have physical interpretations, and can generalize to new parameter regimes. Our framework allows the use of non-convex sparsity promoting regularization functions and can be adapted to address key challenges in scientific problems and data sets, including outliers, parametric dependencies, and physical constraints. We show that the approach discovers parsimonious dynamical models on several example systems. This flexible approach can be tailored to the unique challenges associated with a wide range of applications and data sets, providing a powerful ML-based framework for learning governing models for physical systems from data. △ Less

Submitted 2 July, 2020; v1 submitted 25 June, 2019; originally announced June 2019.

Comments: 22 pages, 5 figures

Discovery of Physics from Data: Universal Laws and Discrepancies

Authors: Brian M. de Silva, David M. Higdon, Steven L. Brunton, J. Nathan Kutz

Abstract: Machine learning (ML) and artificial intelligence (AI) algorithms are now being used to automate the discovery of physics principles and governing equations from measurement data alone. However, positing a universal physical law from data is challenging without simultaneously proposing an accompanying discrepancy model to account for the inevitable mismatch between theory and measurements. By revi… ▽ More Machine learning (ML) and artificial intelligence (AI) algorithms are now being used to automate the discovery of physics principles and governing equations from measurement data alone. However, positing a universal physical law from data is challenging without simultaneously proposing an accompanying discrepancy model to account for the inevitable mismatch between theory and measurements. By revisiting the classic problem of modeling falling objects of different size and mass, we highlight a number of nuanced issues that must be addressed by modern data-driven methods for automated physics discovery. Specifically, we show that measurement noise and complex secondary physical mechanisms, like unsteady fluid drag forces, can obscure the underlying law of gravitation, leading to an erroneous model. We use the sparse identification of nonlinear dynamics (SINDy) method to identify governing equations for real-world measurement data and simulated trajectories. Incorporating into SINDy the assumption that each falling object is governed by a similar physical law is shown to improve the robustness of the learned models, but discrepancies between the predictions and observations persist due to subtleties in drag dynamics. This work highlights the fact that the naive application of ML/AI will generally be insufficient to infer universal physical laws without further modification. △ Less

Submitted 16 April, 2020; v1 submitted 19 June, 2019; originally announced June 2019.

Dynamic Mode Decomposition and Sparse Measurements for Characterization and Monitoring of Power System Disturbances

Authors: J. Jorge Ramos, J. Nathan Kutz

Abstract: We introduce the dynamics mode decomposition for monitoring wide-area power grid networks from sparse measurement data. The mathematical framework fuses data from multiple sensors based on multivariate statistics, providing accurate full state estimation from limited measurements and generating data-driven forecasts for the state of the system. Our proposed data-driven strategy, which is based on… ▽ More We introduce the dynamics mode decomposition for monitoring wide-area power grid networks from sparse measurement data. The mathematical framework fuses data from multiple sensors based on multivariate statistics, providing accurate full state estimation from limited measurements and generating data-driven forecasts for the state of the system. Our proposed data-driven strategy, which is based on energy metrics, can be used for the analysis of major disturbances in the network. The approach is tested and validated using time domain simulations in the IEEE 118 bus system under various disturbance scenarios and under different sparse observations of the system. In addition to state reconstruction, the minimal number of sensors required for monitoring disturbances can be evaluated. Visualization techniques are developed in order to aid in the analysis and characterization of the system after disturbance. △ Less

Submitted 8 June, 2019; originally announced June 2019.

Comments: 11 pages, 10 figures

Data-driven approximations of dynamical systems operators for control

Authors: Eurika Kaiser, J. Nathan Kutz, Steven L. Brunton

Abstract: The Koopman and Perron Frobenius transport operators are fundamentally changing how we approach dynamical systems, providing linear representations for even strongly nonlinear dynamics. Although there is tremendous potential benefit of such a linear representation for estimation and control, transport operators are infinite-dimensional, making them difficult to work with numerically. Obtaining low… ▽ More The Koopman and Perron Frobenius transport operators are fundamentally changing how we approach dynamical systems, providing linear representations for even strongly nonlinear dynamics. Although there is tremendous potential benefit of such a linear representation for estimation and control, transport operators are infinite-dimensional, making them difficult to work with numerically. Obtaining low-dimensional matrix approximations of these operators is paramount for applications, and the dynamic mode decomposition has quickly become a standard numerical algorithm to approximate the Koopman operator. Related methods have seen rapid development, due to a combination of an increasing abundance of data and the extensibility of DMD based on its simple framing in terms of linear algebra. In this chapter, we review key innovations in the data-driven characterization of transport operators for control, providing a high-level and unified perspective. We emphasize important recent developments around sparsity and control, and discuss emerging methods in big data and machine learning. △ Less

Submitted 26 February, 2019; originally announced February 2019.

Comments: 37 pages, 4 figures

Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors

Authors: N. Benjamin Erichson, Lionel Mathelin, Zhewei Yao, Steven L. Brunton, Michael W. Mahoney, J. Nathan Kutz

Abstract: In many applications, it is important to reconstruct a fluid flow field, or some other high-dimensional state, from limited measurements and limited data. In this work, we propose a shallow neural network-based learning methodology for such fluid flow reconstruction. Our approach learns an end-to-end map** between the sensor measurements and the high-dimensional fluid flow field, without any hea… ▽ More In many applications, it is important to reconstruct a fluid flow field, or some other high-dimensional state, from limited measurements and limited data. In this work, we propose a shallow neural network-based learning methodology for such fluid flow reconstruction. Our approach learns an end-to-end map** between the sensor measurements and the high-dimensional fluid flow field, without any heavy preprocessing on the raw data. No prior knowledge is assumed to be available, and the estimation method is purely data-driven. We demonstrate the performance on three examples in fluid mechanics and oceanography, showing that this modern data-driven approach outperforms traditional modal approximation techniques which are commonly used for flow reconstruction. Not only does the proposed method show superior performance characteristics, it can also produce a comparable level of performance with traditional methods in the area, using significantly fewer sensors. Thus, the mathematical architecture is ideal for emerging global monitoring technologies where measurement data are often limited. △ Less

Submitted 27 December, 2020; v1 submitted 19 February, 2019; originally announced February 2019.

Journal ref: Proc. R. Soc. A476: 20200097 (2020)

Engineering Structural Robustness in Power Grid Networks Susceptible to Coherent Swing Instability

Authors: Daniel Dylewsky, Xiu Yang, Alexandre Tartakovsky, J. Nathan Kutz

Abstract: Networked power grid systems are susceptible to a phenomenon known as Coherent Swing Instability (CSI), in which a subset of machines in the grid lose synchrony with the rest of the network. We develop network level evaluation metrics to (i) identify community substructures in the power grid network, (ii) determine weak points in the network that are particularly sensitive to CSI, and (iii) produc… ▽ More Networked power grid systems are susceptible to a phenomenon known as Coherent Swing Instability (CSI), in which a subset of machines in the grid lose synchrony with the rest of the network. We develop network level evaluation metrics to (i) identify community substructures in the power grid network, (ii) determine weak points in the network that are particularly sensitive to CSI, and (iii) produce an engineering approach for the addition of transmission lines to reduce the incidences of CSI in existing networks, or design new power grid networks that are robust to CSI by their network design. For simulations on a reduced model for the American Northeast power grid, where a block of buses representing the New England region exhibit a strong propensity for CSI, we show that modifying the network's connectivity structure can markedly improve the grid's resilience to CSI. Our analysis provides a versatile diagnostic tool for evaluating the efficacy of adding lines to a power grid which is known to be prone to CSI. This is a particularly relevant problem in large-scale power systems, where improving stability and robustness to interruptions by increasing overall network connectivity is not feasible due to financial and infrastructural constraints. △ Less

Submitted 22 May, 2019; v1 submitted 24 May, 2018; originally announced May 2018.

Comments: 15 pages, 10 figures

Journal ref: Applied Network Science 4(1) 24 (2019)

Sparse Identification of Nonlinear Dynamics for Rapid Model Recovery

Authors: Markus Quade, Markus Abel, J. Nathan Kutz, Steven L. Brunton

Abstract: Big data has become a critically enabling component of emerging mathematical methods aimed at the automated discovery of dynamical systems, where first principles modeling may be intractable. However, in many engineering systems, abrupt changes must be rapidly characterized based on limited, incomplete, and noisy data. Many leading automated learning techniques rely on unrealistically large data s… ▽ More Big data has become a critically enabling component of emerging mathematical methods aimed at the automated discovery of dynamical systems, where first principles modeling may be intractable. However, in many engineering systems, abrupt changes must be rapidly characterized based on limited, incomplete, and noisy data. Many leading automated learning techniques rely on unrealistically large data sets and it is unclear how to leverage prior knowledge effectively to re-identify a model after an abrupt change. In this work, we propose a conceptual framework to recover parsimonious models of a system in response to abrupt changes in the low-data limit. First, the abrupt change is detected by comparing the estimated Lyapunov time of the data with the model prediction. Next, we apply the sparse identification of nonlinear dynamics (SINDy) regression to update a previously identified model with the fewest changes, either by addition, deletion, or modification of existing model terms. We demonstrate this sparse model recovery on several examples for abrupt system change detection in periodic and chaotic dynamical systems. Our examples show that sparse updates to a previously identified model perform better with less data, have lower runtime complexity, and are less sensitive to noise than identifying an entirely new model. The proposed abrupt-SINDy architecture provides a new paradigm for the rapid and efficient recovery of a system model after abrupt changes. △ Less

Submitted 3 June, 2018; v1 submitted 2 March, 2018; originally announced March 2018.

Optimized Sampling for Multiscale Dynamics

Authors: Krithika Manohar, Eurika Kaiser, Steven L. Brunton, J. Nathan Kutz

Abstract: The characterization of intermittent, multiscale and transient dynamics using data-driven analysis remains an open challenge. We demonstrate an application of the Dynamic Mode Decomposition (DMD) with sparse sampling for the diagnostic analysis of multiscale physics. The DMD method is an ideal spatiotemporal matrix decomposition that correlates spatial features of computational or experimental dat… ▽ More The characterization of intermittent, multiscale and transient dynamics using data-driven analysis remains an open challenge. We demonstrate an application of the Dynamic Mode Decomposition (DMD) with sparse sampling for the diagnostic analysis of multiscale physics. The DMD method is an ideal spatiotemporal matrix decomposition that correlates spatial features of computational or experimental data to periodic temporal behavior. DMD can be modified into a multiresolution analysis to separate complex dynamics into a hierarchy of multiresolution timescale components, where each level of the hierarchy divides dynamics into distinct background (slow) and foreground (fast) timescales. The multiresolution DMD is capable of characterizing nonlinear dynamical systems in an equation-free manner by recursively decomposing the state of the system into low-rank spatial modes and their temporal Fourier dynamics. Moreover, these multiresolution DMD modes can be used to determined sparse sampling locations which are nearly optimal for dynamic regime classification and full state reconstruction. Specifically, optimized sensors are efficiently chosen using QR column pivots of the DMD library, thus avoiding an NP-hard selection process. We demonstrate the efficacy of the method on several examples, including global sea-surface temperature data, and show that only a small number of sensors are needed for accurate global reconstructions and classification of El Niño events. △ Less

Submitted 13 December, 2017; originally announced December 2017.

Comments: 19 pages, 10 figures

Journal ref: Multiscale Model. Simul. 17 (2019) 117-136