Showing 1–40 of 40 results for author: Yeung, E

Demonstration of MaskSearch: Efficiently Querying Image Masks for Machine Learning Workflows

Authors: Lindsey Linxi Wei, Chung Yik Edward Yeung, Hongjian Yu, **gchuan Zhou, Dong He, Magdalena Balazinska

Abstract: We demonstrate MaskSearch, a system designed to accelerate queries over databases of image masks generated by machine learning models. MaskSearch formalizes and accelerates a new category of queries for retrieving images and their corresponding masks based on mask properties, which support various applications, from identifying spurious correlations learned by models to exploring discrepancies bet… ▽ More We demonstrate MaskSearch, a system designed to accelerate queries over databases of image masks generated by machine learning models. MaskSearch formalizes and accelerates a new category of queries for retrieving images and their corresponding masks based on mask properties, which support various applications, from identifying spurious correlations learned by models to exploring discrepancies between model saliency and human attention. This demonstration makes the following contributions:(1) the introduction of MaskSearch's graphical user interface (GUI), which enables interactive exploration of image databases through mask properties, (2) hands-on opportunities for users to explore MaskSearch's capabilities and constraints within machine learning workflows, and (3) an opportunity for conference attendees to understand how MaskSearch accelerates queries over image masks. △ Less

Submitted 9 April, 2024; originally announced April 2024.

An SVD-like Decomposition of Bounded-Input Bounded-Output Functions

Authors: Brian Charles Brown, Michael King, Sean Warnick, Enoch Yeung, David Grimsman

Abstract: The Singular Value Decomposition (SVD) of linear functions facilitates the calculation of their 2-induced norm and row and null spaces, hallmarks of linear control theory. In this work, we present a function representation that, similar to SVD, provides an upper bound on the 2-induced norm of bounded-input bounded-output functions, as well as facilitates the computation of generalizations of the n… ▽ More The Singular Value Decomposition (SVD) of linear functions facilitates the calculation of their 2-induced norm and row and null spaces, hallmarks of linear control theory. In this work, we present a function representation that, similar to SVD, provides an upper bound on the 2-induced norm of bounded-input bounded-output functions, as well as facilitates the computation of generalizations of the notions of row and null spaces. Borrowing from the notion of "lifting" in Koopman operator theory, we construct a finite-dimensional lifting of inputs that relaxes the unitary property of the right-most matrix in traditional SVD, $V^*$, to be an injective, norm-preserving map** to a slightly higher-dimensional space. △ Less

Submitted 29 March, 2024; originally announced April 2024.

Comments: Submitted to the 2024 Conference on Decision and Control

Validation of Consumer-grade Digital Camera-based Human Activity Evaluation for Upper Limb Exercises and Development of a Therapist-guided, Automated Telerehabilitation Framework and Platform for Stroke Rehabilitation

Authors: Elton H. L. Yeung, Yingxian Chen, Wilton W. T. Fok, Gary K. K. Lau

Abstract: Timely and adequate rehabilitation is critical in facilitating post-stroke recovery. However, the organization and delivery of rehabilitation are resource-demanding, and are only available to approximately 25% of stroke survivors in low-to-middle-income countries. Improving access to stroke rehabilitation services through innovative solutions is therefore urgently required. Tele-rehabilitation, wh… ▽ More Timely and adequate rehabilitation is critical in facilitating post-stroke recovery. However, the organization and delivery of rehabilitation are resource-demanding, and are only available to approximately 25% of stroke survivors in low-to-middle-income countries. Improving access to stroke rehabilitation services through innovative solutions is therefore urgently required. Tele-rehabilitation, which transits care to home- and community settings, has emerged as a promising solution. However, current approaches using video tutorial, teleconference, or other specialized devices face inherent shortfalls that limit their uptake. In this study, we proposed and validated the use of an open-source, markerless motion capture model with consumer-grade devices to overcome these challenges. Our solution enables reliable measurement of the end range of motion during upper limb exercises with near-perfect waveform similarity and intraclass correlation to that of the gold standard Kinect approach. Our multidisciplinary team developed an automated telerehabilitation framework incorporating the validated markerless technique to facilitate a seamless telerehabilitation process. It enables personalized rehabilitation plans with real-time feedback, and individual progress reports using objective quantitative and qualitative features to improve patient monitoring and management, and home-based rehabilitation service uptake and compliance. This study serves as a proof-of-concept in preparation for the future development of a detailed model of care, and feasibility, usability, and cost-effectiveness studies of an automated telerehabilitation platform and framework in improving the state of post-stroke rehabilitation and functional outcome. △ Less

Submitted 10 February, 2024; v1 submitted 21 November, 2023; originally announced November 2023.

Comments: it's not a ready paper to be uploaded

An Algorithm for Modelling Escalator Fixed Loss Energy for PHM and sustainable energy usage

Authors: Xuwen Hu, Jiaqi Qiu, Yu Lin, Inez Maria Zwetsloot, William Ka Fai Lee, Edmond Yin San Yeung, Colman Yiu Wah Yeung, Chris Chun Long Wong

Abstract: Prognostic Health Management (PHM) is designed to assess and monitor the health status of systems, anticipate the onset of potential failure, and prevent unplanned downtime. In recent decades, collecting massive amounts of real-time sensor data enabled condition monitoring (CM) and consequently, detection of abnormalities to support maintenance decision-making. Additionally, the utilization of PHM… ▽ More Prognostic Health Management (PHM) is designed to assess and monitor the health status of systems, anticipate the onset of potential failure, and prevent unplanned downtime. In recent decades, collecting massive amounts of real-time sensor data enabled condition monitoring (CM) and consequently, detection of abnormalities to support maintenance decision-making. Additionally, the utilization of PHM techniques can support energy sustainability efforts by optimizing energy usage and identifying opportunities for energy-saving measures. Escalators are efficient machines for transporting people and goods, and measuring energy consumption in time can facilitate PHM of escalators. Fixed loss energy, or no-load energy, of escalators denotes the energy consumption by an unloaded escalator. Fixed loss energy varies over time indicating varying operating conditions. In this paper, we propose to use escalators' fixed loss energy for PHM. We propose an approach to compute daily fixed loss energy based on energy consumption sensor data. The proposed approach is validated using a set of experimental data. The advantages and disadvantages of each approach are also presented, and recommendations are given. Finally, to illustrate PHM, we set up an EWMA chart for monitoring the fixed loss over time and demonstrate the potential in reducing energy costs associated with escalator operation. △ Less

Submitted 6 September, 2023; originally announced September 2023.

On-chip indistinguishable photons using III-V nanowire/SiN hybrid integration

Authors: Edith Yeung, David B. Northeast, Jeongwan **, Patrick Laferrière, Marek Korkusinski, Philip J. Poole, Robin L. Williams, Dan Dalacu

Abstract: We demonstrate on-chip generation of indistinguishable photons based on a nanowire quantum dot. From a growth substrate containing arrays of positioned-controlled single dot nanowires, we select a single nanowire which is placed on a SiN waveguide fabricated on a Si-based chip. Coupling of the quantum dot emission to the SiN waveguide is via the evanescent mode in the tapered nanowire. Post-select… ▽ More We demonstrate on-chip generation of indistinguishable photons based on a nanowire quantum dot. From a growth substrate containing arrays of positioned-controlled single dot nanowires, we select a single nanowire which is placed on a SiN waveguide fabricated on a Si-based chip. Coupling of the quantum dot emission to the SiN waveguide is via the evanescent mode in the tapered nanowire. Post-selected two-photon interference visibilities using continuous wave excitation above-band and into a p-shell of the dot were 100%, consistent with a single photon source having negligible multi-photon emission probability. Visibilities over the entire photon wavepacket, measured using pulsed excitation, were reduced by a factor of 5 when exciting quasi-resonantly and by a factor of 10 for above-band excitation. The role of excitation timing jitter, spectral diffusion and pure dephasing in limiting visibilities over the temporal extent of the photon is investigated using additional measurements of the coherence and linewidth of the emitted photons. △ Less

Submitted 20 August, 2023; originally announced August 2023.

Remaining Useful Life Modelling with an Escalator Health Condition Analytic System

Authors: Inez M. Zwetsloot, Yu Lin, Jiaqi Qiu, Lishuai Li, William Ka Fai Lee, Edmond Yin San Yeung, Colman Yiu Wah Yeung, Chris Chun Long Wong

Abstract: The refurbishment of an escalator is usually linked with its design life as recommended by the manufacturer. However, the actual useful life of an escalator should be determined by its operating condition which is affected by the runtime, workload, maintenance quality, vibration, etc., rather than age only. The objective of this project is to develop a comprehensive health condition analytic syste… ▽ More The refurbishment of an escalator is usually linked with its design life as recommended by the manufacturer. However, the actual useful life of an escalator should be determined by its operating condition which is affected by the runtime, workload, maintenance quality, vibration, etc., rather than age only. The objective of this project is to develop a comprehensive health condition analytic system for escalators to support refurbishment decisions. The analytic system consists of four parts: 1) online data gathering and processing; 2) a dashboard for condition monitoring; 3) a health index model; and 4) remaining useful life prediction. The results can be used for a) predicting the remaining useful life of the escalators, in order to support asset replacement planning and b) monitoring the real-time condition of escalators; including alerts when vibration exceeds the threshold and signal diagnosis, giving an indication of possible root cause (components) of the alert signal. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Comments: 14 pages, 12 figures, 7 tables

Learning Invariant Subspaces of Koopman Operators--Part 2: Heterogeneous Dictionary Mixing to Approximate Subspace Invariance

Authors: Charles A. Johnson, Shara Balakrishnan, Enoch Yeung

Abstract: This work builds on the models and concepts presented in part 1 to learn approximate dictionary representations of Koopman operators from data. Part I of this paper presented a methodology for arguing the subspace invariance of a Koopman dictionary. This methodology was demonstrated on the state-inclusive logistic lifting (SILL) basis. This is an affine basis augmented with conjunctive logistic fu… ▽ More This work builds on the models and concepts presented in part 1 to learn approximate dictionary representations of Koopman operators from data. Part I of this paper presented a methodology for arguing the subspace invariance of a Koopman dictionary. This methodology was demonstrated on the state-inclusive logistic lifting (SILL) basis. This is an affine basis augmented with conjunctive logistic functions. The SILL dictionary's nonlinear functions are homogeneous, a norm in data-driven dictionary learning of Koopman operators. In this paper, we discover that structured mixing of heterogeneous dictionary functions drawn from different classes of nonlinear functions achieve the same accuracy and dimensional scaling as the deep-learning-based deepDMD algorithm. We specifically show this by building a heterogeneous dictionary comprised of SILL functions and conjunctive radial basis functions (RBFs). This mixed dictionary achieves the same accuracy and dimensional scaling as deepDMD with an order of magnitude reduction in parameters, while maintaining geometric interpretability. These results strengthen the viability of dictionary-based Koopman models to solving high-dimensional nonlinear learning problems. △ Less

Submitted 14 December, 2022; originally announced December 2022.

Comments: 16 pages, 6 figures. arXiv admin note: substantial text overlap with arXiv:2206.13585

Learning Invariant Subspaces of Koopman Operators--Part 1: A Methodology for Demonstrating a Dictionary's Approximate Subspace Invariance

Authors: Charles A. Johnson, Shara Balakrishnan, Enoch Yeung

Abstract: Koopman operators model nonlinear dynamics as a linear dynamic system acting on a nonlinear function as the state. This nonstandard state is often called a Koopman observable and is usually approximated numerically by a superposition of functions drawn from a dictionary. In a widely used algorithm, Extended Dynamic Mode Decomposition, the dictionary functions are drawn from a fixed class of functi… ▽ More Koopman operators model nonlinear dynamics as a linear dynamic system acting on a nonlinear function as the state. This nonstandard state is often called a Koopman observable and is usually approximated numerically by a superposition of functions drawn from a dictionary. In a widely used algorithm, Extended Dynamic Mode Decomposition, the dictionary functions are drawn from a fixed class of functions. Recently, deep learning combined with EDMD has been used to learn novel dictionary functions in an algorithm called deep dynamic mode decomposition (deepDMD). The learned representation both (1) accurately models and (2) scales well with the dimension of the original nonlinear system. In this paper we analyze the learned dictionaries from deepDMD and explore the theoretical basis for their strong performance. We explore State-Inclusive Logistic Lifting (SILL) dictionary functions to approximate Koopman observables. Error analysis of these dictionary functions show they satisfy a property of subspace approximation, which we define as uniform finite approximate closure. Our results provide a hypothesis to explain the success of deep neural networks in learning numerical approximations to Koopman operators. Part 2 of this paper will extend this explanation by demonstrating the subspace invariant of heterogeneous dictionaries and presenting a head-to-head numerical comparison of deepDMD and low-parameter heterogeneous dictionary learning. △ Less

Submitted 14 December, 2022; originally announced December 2022.

Comments: 13 pages, 2 figures. arXiv admin note: substantial text overlap with arXiv:2206.13585

Data-Driven Observability Decomposition with Koopman Operators for Optimization of Output Functions of Nonlinear Systems

Authors: Shara Balakrishnan, Aqib Hasnain, Robert Egbert, Enoch Yeung

Abstract: When complex systems with nonlinear dynamics achieve an output performance objective, only a fraction of the state dynamics significantly impacts that output. Those minimal state dynamics can be identified using the differential geometric approach to the observability of nonlinear systems, but the theory is limited to only analytical systems. In this paper, we extend the notion of nonlinear observ… ▽ More When complex systems with nonlinear dynamics achieve an output performance objective, only a fraction of the state dynamics significantly impacts that output. Those minimal state dynamics can be identified using the differential geometric approach to the observability of nonlinear systems, but the theory is limited to only analytical systems. In this paper, we extend the notion of nonlinear observable decomposition to the more general class of data-informed systems. We employ Koopman operator theory, which encapsulates nonlinear dynamics in linear models, allowing us to bridge the gap between linear and nonlinear observability notions. We propose a new algorithm to learn Koopman operator representations that capture the system dynamics while ensuring that the output performance measure is in the span of its observables. We show that a transformation of this linear, output-inclusive Koopman model renders a new minimum Koopman representation. This representation embodies only the observable portion of the nonlinear observable decomposition of the original system. A prime application of this theory is to identify genes in biological systems that correspond to specific phenotypes, the performance measure. We simulate two biological gene networks and demonstrate that the observability of Koopman operators can successfully identify genes that drive each phenotype. We anticipate our novel system identification tool will effectively discover reduced gene networks that drive complex behaviors in biological systems. △ Less

Submitted 17 October, 2022; originally announced October 2022.

Comments: 17 pages, 4 figures, submission to Automatica

Approaching transform-limited photons from nanowire quantum dots excited above-band

Authors: Patrick Laferrière, Aria Yin, Edith Yeung, Leila Kusmic, Marek Korkusinski, Payman Rasekh, David B. Northeast, Sofiane Haffouz, Jean Lapointe, Philip J. Poole, Robin L. Williams, Dan Dalacu

Abstract: We demonstrate that, even when employing above-band excitation, photons emitted from semiconductor quantum dots can have linewidths that approach their transform-limited values. This is accomplished by using quantum dots embedded in bottom-up photonic nanowires, an approach which mitigates several potential mechanisms that can result in linewidth broadening: (i) only a single quantum dot is presen… ▽ More We demonstrate that, even when employing above-band excitation, photons emitted from semiconductor quantum dots can have linewidths that approach their transform-limited values. This is accomplished by using quantum dots embedded in bottom-up photonic nanowires, an approach which mitigates several potential mechanisms that can result in linewidth broadening: (i) only a single quantum dot is present in each device, (ii) dot nucleation proceeds without the formation of a wetting layer, and (iii) the sidewalls of the photonic nanowire are comprised not of etched facets, but of epitaxially grown crystal planes. Using these structures we achieve linewidths of 2x the transform limit, unprecedented for above-band excitation. We also demonstrate a highly nonlinear dependence of the linewidth on both excitation power and temperature which can be described by an independent Boson model that considers both deformation and piezoelectric exciton-phonon coupling. We find that for sufficiently low excitation powers and temperatures, the observed excess broadening is not dominated by phonon dephasing, a surprising result considering the high phonon occupation that occurs with above-band excitation. △ Less

Submitted 29 July, 2022; originally announced August 2022.

Journal ref: Phys. Rev. B 107, (2023) 15522

Heterogeneous mixtures of dictionary functions to approximate subspace invariance in Koopman operators

Authors: Charles A. Johnson, Shara Balakrishnan, Enoch Yeung

Abstract: Koopman operators model nonlinear dynamics as a linear dynamic system acting on a nonlinear function as the state. This nonstandard state is often called a Koopman observable and is usually approximated numerically by a superposition of functions drawn from a \textit{dictionary}. A widely used algorithm, is \textit{Extended Dynamic Mode Decomposition}, where the dictionary functions are drawn from… ▽ More Koopman operators model nonlinear dynamics as a linear dynamic system acting on a nonlinear function as the state. This nonstandard state is often called a Koopman observable and is usually approximated numerically by a superposition of functions drawn from a \textit{dictionary}. A widely used algorithm, is \textit{Extended Dynamic Mode Decomposition}, where the dictionary functions are drawn from a fixed, homogeneous class of functions. Recently, deep learning combined with EDMD has been used to learn novel dictionary functions in an algorithm called deep dynamic mode decomposition (deepDMD). The learned representation both (1) accurately models and (2) scales well with the dimension of the original nonlinear system. In this paper we analyze the learned dictionaries from deepDMD and explore the theoretical basis for their strong performance. We discover a novel class of dictionary functions to approximate Koopman observables. Error analysis of these dictionary functions show they satisfy a property of subspace approximation, which we define as uniform finite approximate closure. We discover that structured mixing of heterogeneous dictionary functions drawn from different classes of nonlinear functions achieve the same accuracy and dimensional scaling as deepDMD. This mixed dictionary does so with an order of magnitude reduction in parameters, while maintaining geometric interpretability. Our results provide a hypothesis to explain the success of deep neural networks in learning numerical approximations to Koopman operators. △ Less

Submitted 27 June, 2022; originally announced June 2022.

Comments: 14 pages, 7 figures, journal paper

On the Stability, Economic Efficiency and Incentive Compatibility of Electricity Market Dynamics

Authors: Pengcheng You, Yan Jiang, Enoch Yeung, Dennice F. Gayme, Enrique Mallada

Abstract: This paper focuses on the operation of an electricity market that accounts for participants that bid at a sub-minute timescale. To that end, we model the market-clearing process as a dynamical system, called market dynamics, which is temporally coupled with the grid frequency dynamics and is thus required to guarantee system-wide stability while meeting the system operational constraints. We chara… ▽ More This paper focuses on the operation of an electricity market that accounts for participants that bid at a sub-minute timescale. To that end, we model the market-clearing process as a dynamical system, called market dynamics, which is temporally coupled with the grid frequency dynamics and is thus required to guarantee system-wide stability while meeting the system operational constraints. We characterize participants as price-takers who rationally update their bids to maximize their utility in response to real-time schedules of prices and dispatch. For two common bidding mechanisms, based on quantity and price, we identify a notion of alignment between participants' behavior and planners' goals that leads to a saddle-based design of the market that guarantees convergence to a point meeting all operational constraints. We further explore cases where this alignment property does not hold and observe that misaligned participants' bidding can destabilize the closed-loop system. We thus design a regularized version of the market dynamics that recovers all the desirable stability and steady-state performance guarantees. Numerical tests validate our results on the IEEE 39-bus system. △ Less

Submitted 10 December, 2021; originally announced December 2021.

Unity yield of deterministically positioned quantum dot single photon sources

Authors: Patrick Laferrière, Edith Yeung, Isabelle Miron, David B. Northeast, Sofiane Haffouz, Jean Lapointe, Marek Korkusinski, Philip J. Poole, Robin L. Williams, Dan Dalacu

Abstract: We report on a platform for the production of single photon devices with a fabrication yield of 100%. The sources are based on InAsP quantum dots embedded within position-controlled bottom-up InP nanowires. Using optimized growth conditions, we produce large arrays of structures having highly uniform geometries. Collection efficiencies are as high as 83% and multiphoton emission probabilities as l… ▽ More We report on a platform for the production of single photon devices with a fabrication yield of 100%. The sources are based on InAsP quantum dots embedded within position-controlled bottom-up InP nanowires. Using optimized growth conditions, we produce large arrays of structures having highly uniform geometries. Collection efficiencies are as high as 83% and multiphoton emission probabilities as low as 0.6% with the distribution away from optimal values associated with the excitation of other charge complexes and re-excitation processes, respectively, inherent to the above-band excitation employed. Importantly, emission peak lineshapes have Lorentzian profiles indicating that linewidths are not limited by inhomogeneous broadening but rather pure dephasing, likely elastic carrier-phonon scattering due to a high phonon occupation. This work establishes nanowire-based devices as a viable route for the scalable fabrication of efficient single photon sources and provides a valuable resource for hybrid on-chip platforms currently being developed. △ Less

Submitted 15 October, 2021; originally announced October 2021.

Comments: 9 pages, 5 figures

Data-Driven Operator Theoretic Methods for Phase Space Learning and Analysis

Authors: Sai Pushpak Nandanoori, Subhrajit Sinha, Enoch Yeung

Abstract: This paper uses data-driven operator theoretic approaches to explore the global phase space of a dynamical system. We defined conditions for discovering new invariant subspaces in the state space of a dynamical system starting from an invariant subspace based on the spectral properties of the Koopman operator. When the system evolution is known locally in several invariant subspaces in the state s… ▽ More This paper uses data-driven operator theoretic approaches to explore the global phase space of a dynamical system. We defined conditions for discovering new invariant subspaces in the state space of a dynamical system starting from an invariant subspace based on the spectral properties of the Koopman operator. When the system evolution is known locally in several invariant subspaces in the state space of a dynamical system, a phase space stitching result is derived that yields the global Koopman operator. Additionally, in the case of equivariant systems, a phase space stitching result is developed to identify the global Koopman operator using the symmetry properties between the invariant subspaces of the dynamical system and time-series data from any one of the invariant subspaces. Finally, these results are extended to topologically conjugate dynamical systems; in particular, the relation between the Koopman tuple of topologically conjugate systems is established. The proposed results are demonstrated on several second-order nonlinear dynamical systems including a bistable toggle switch. Our method elucidates a strategy for designing discovery experiments: experiment execution can be done in many steps, and models from different invariant subspaces can be combined to approximate the global Koopman operator. △ Less

Submitted 29 June, 2021; originally announced June 2021.

Comments: 32 pages, 19 figures. This paper is currently under review

The Effect of Sensor Fusion on Data-Driven Learning of Koopman Operators

Authors: Shara Balakrishnan, Aqib Hasnain, Rob Egbert, Enoch Yeung

Abstract: Dictionary methods for system identification typically rely on one set of measurements to learn governing dynamics of a system. In this paper, we investigate how fusion of output measurements with state measurements affects the dictionary selection process in Koopman operator learning problems. While prior methods use dynamical conjugacy to show a direct link between Koopman eigenfunctions in two… ▽ More Dictionary methods for system identification typically rely on one set of measurements to learn governing dynamics of a system. In this paper, we investigate how fusion of output measurements with state measurements affects the dictionary selection process in Koopman operator learning problems. While prior methods use dynamical conjugacy to show a direct link between Koopman eigenfunctions in two distinct data spaces (measurement channels), we explore the specific case where output measurements are nonlinear, non-invertible functions of the system state. This setup reflects the measurement constraints of many classes of physical systems, e.g., biological measurement data, where one type of measurement does not directly transform to another. We propose output constrained Koopman operators (OC-KOs) as a new framework to fuse two measurement sets. We show that OC-KOs are effective for sensor fusion by proving that when learning a Koopman operator, output measurement functions serve to constrain the space of potential Koopman observables and their eigenfunctions. Further, low-dimensional output measurements can be embedded to inform selection of Koopman dictionary functions for high-dimensional models. We propose two algorithms to identify OC-KO representations directly from data: a direct optimization method that uses state and output data simultaneously and a sequential optimization method. We prove a theorem to show that the solution spaces of the two optimization problems are equivalent. We illustrate these findings with a theoretical example and two numerical simulations. △ Less

Submitted 29 June, 2021; originally announced June 2021.

Comments: 20 pages, 7 figures

On Few Shot Learning of Dynamical Systems: A Koopman Operator Theoretic Approach

Authors: Subhrajit Sinha, Umesh Vaidya, Enoch Yeung

Abstract: In this paper, we propose a novel algorithm for learning the Koopman operator of a dynamical system from a \textit{small} amount of training data. In many applications of data-driven modeling, e.g. biological network modeling, cybersecurity, modeling the Internet of Things, or smart grid monitoring, it is impossible to obtain regularly sampled time-series data with a sufficiently high sampling fre… ▽ More In this paper, we propose a novel algorithm for learning the Koopman operator of a dynamical system from a \textit{small} amount of training data. In many applications of data-driven modeling, e.g. biological network modeling, cybersecurity, modeling the Internet of Things, or smart grid monitoring, it is impossible to obtain regularly sampled time-series data with a sufficiently high sampling frequency. In such situations the existing Dynamic Mode Decomposition (DMD) or Extended Dynamic Mode Decomposition (EDMD) algorithms for Koopman operator computation often leads to a low fidelity approximate Koopman operator. To this end, this paper proposes an algorithm which can compute the Koopman operator efficiently when the training data-set is sparsely sampled across time. In particular, the proposed algorithm enriches the small training data-set by appending artificial data points, which are treated as noisy observations. The larger, albeit noisy data-set is then used to compute the Koopman operator, using techniques from Robust Optimization. The efficacy of the proposed algorithm is also demonstrated on three different dynamical systems, namely a linear network of oscillators, a nonlinear system and a dynamical system governed by a Partial Differential Equation (PDE). △ Less

Submitted 6 March, 2021; originally announced March 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:1901.03024

Computationally Efficient Learning of Large Scale Dynamical Systems: A Koopman Theoretic Approach

Authors: Subhrajit Sinha, Sai Pushpak Nandanoori, Enoch Yeung

Abstract: In recent years there has been a considerable drive towards data-driven analysis, discovery and control of dynamical systems. To this end, operator theoretic methods, namely, Koopman operator methods have gained a lot of interest. In general, the Koopman operator is obtained as a solution to a least-squares problem, and as such, the Koopman operator can be expressed as a closed-form solution that… ▽ More In recent years there has been a considerable drive towards data-driven analysis, discovery and control of dynamical systems. To this end, operator theoretic methods, namely, Koopman operator methods have gained a lot of interest. In general, the Koopman operator is obtained as a solution to a least-squares problem, and as such, the Koopman operator can be expressed as a closed-form solution that involves the computation of Moore-Penrose inverse of a matrix. For high dimensional systems and also if the size of the obtained data-set is large, the computation of the Moore-Penrose inverse becomes computationally challenging. In this paper, we provide an algorithm for computing the Koopman operator for high dimensional systems in a time-efficient manner. We further demonstrate the efficacy of the proposed approach on two different systems, namely a network of coupled oscillators (with state-space dimension up to 2500) and IEEE 68 bus system (with state-space dimension 204 and up to 24,000 time-points). △ Less

Submitted 1 July, 2020; originally announced July 2020.

Prediction of fitness in bacteria with causal jump dynamic mode decomposition

Authors: Shara Balakrishnan, Aqib Hasnain, Nibodh Boddupalli, Dennis M. Joshy, Robert G. Egbert, Enoch Yeung

Abstract: In this paper, we consider the problem of learning a predictive model for population cell growth dynamics as a function of the media conditions. We first introduce a generic data-driven framework for training operator-theoretic models to predict cell growth rate. We then introduce the experimental design and data generated in this study, namely growth curves of Pseudomonas putida as a function of… ▽ More In this paper, we consider the problem of learning a predictive model for population cell growth dynamics as a function of the media conditions. We first introduce a generic data-driven framework for training operator-theoretic models to predict cell growth rate. We then introduce the experimental design and data generated in this study, namely growth curves of Pseudomonas putida as a function of casein and glucose concentrations. We use a data driven approach for model identification, specifically the nonlinear autoregressive (NAR) model to represent the dynamics. We show theoretically that Hankel DMD can be used to obtain a solution of the NAR model. We show that it identifies a constrained NAR model and to obtain a more general solution, we define a causal state space system using 1-step,2-step,...,τ-step predictors of the NAR model and identify a Koopman operator for this model using extended dynamic mode decomposition. The hybrid scheme we call causal-jump dynamic mode decomposition, which we illustrate on a growth profile or fitness prediction challenge as a function of different input growth conditions. We show that our model is able to recapitulate training growth curve data with 96.6% accuracy and predict test growth curve data with 91% accuracy. △ Less

Submitted 22 June, 2020; originally announced June 2020.

Data Driven Online Learning of Power System Dynamics

Authors: Subhrajit Sinha, Sai Pushpak Nandanoori, Enoch Yeung

Abstract: With the advancement of sensing and communication in power networks, high-frequency real-time data from a power network can be used as a resource to develop better monitoring capabilities. In this work, a systematic approach based on data-driven operator theoretic methods involving Koopman operator is proposed for the online identification of power system dynamics. In particular, a new algorithm i… ▽ More With the advancement of sensing and communication in power networks, high-frequency real-time data from a power network can be used as a resource to develop better monitoring capabilities. In this work, a systematic approach based on data-driven operator theoretic methods involving Koopman operator is proposed for the online identification of power system dynamics. In particular, a new algorithm is provided, which unlike any previously existing algorithms, updates the Koopman operator iteratively as new data points are acquired. The proposed algorithm has three advantages: a) allows for real-time monitoring of the power system dynamics b) linear power system dynamics (this linear system is usually in a higher dimensional feature space and is not same as linearization of the underlying nonlinear dynamics) and c) computationally fast and less intensive when compared to the popular Extended Dynamic Mode Decomposition (EDMD) algorithm. The efficiency of the proposed algorithm is illustrated on an IEEE 9 bus system using synthetic data from the nonlinear model and on IEEE 39 bus system using synthetic data from the linearized model. △ Less

Submitted 10 March, 2020; originally announced March 2020.

Comments: arXiv admin note: substantial text overlap with arXiv:1909.12520

Koopman Operator Methods for Global Phase Space Exploration of Equivariant Dynamical Systems

Authors: Subhrajit Sinha, Sai P. Nandanoori, Enoch Yeung

Abstract: In this paper, we develop the Koopman operator theory for dynamical systems with symmetry. In particular, we investigate how the Koopman operator and eigenfunctions behave under the action of the symmetry group of the underlying dynamical system. Further, exploring the underlying symmetry, we propose an algorithm to construct a global Koopman operator from local Koopman operators. In particular, w… ▽ More In this paper, we develop the Koopman operator theory for dynamical systems with symmetry. In particular, we investigate how the Koopman operator and eigenfunctions behave under the action of the symmetry group of the underlying dynamical system. Further, exploring the underlying symmetry, we propose an algorithm to construct a global Koopman operator from local Koopman operators. In particular, we show, by exploiting the symmetry, data from all the invariant sets are not required for constructing the global Koopman operator; that is, local knowledge of the system is enough to infer the global dynamics. △ Less

Submitted 10 March, 2020; originally announced March 2020.

Data-Driven Operator Theoretic Methods for Global Phase Space Learning

Authors: Sai Pushpak Nandanoori, Subhrajit Sinha, Enoch Yeung

Abstract: In this work, we propose to apply the recently developed Koopman operator techniques to explore the global phase space of a nonlinear system from time-series data. In particular, we address the problem of identifying various invariant subsets of a phase space from the spectral properties of the associated Koopman operator constructed from time-series data. Moreover, in the case when the system evo… ▽ More In this work, we propose to apply the recently developed Koopman operator techniques to explore the global phase space of a nonlinear system from time-series data. In particular, we address the problem of identifying various invariant subsets of a phase space from the spectral properties of the associated Koopman operator constructed from time-series data. Moreover, in the case when the system evolution is known locally in various invariant subspaces, then a phase space stitching result is proposed that can be applied to identify a global Koopman operator. A biological system, bistable toggle switch and a second-order nonlinear system example are considered to illustrate the proposed results. The construction of this global Koopman operator is very helpful in experimental works as multiple experiments can't be run at the same time starting from several initial conditions. △ Less

Submitted 3 October, 2019; originally announced October 2019.

Comments: This version is currently under review in ACC 2020

Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm

Authors: Zhiyuan Liu, Guohui Ding, Lijun Chen, Enoch Yeung

Abstract: We propose an alternating optimization algorithm to the nonconvex Koopman operator learning problem for nonlinear dynamic systems. We show that the proposed algorithm will converge to a critical point with rate $O(1/T)$ and $O(\frac{1}{\log T})$ for the constant and diminishing learning rates, respectively, under some mild conditions. To cope with the high dimensional nonlinear dynamical systems,… ▽ More We propose an alternating optimization algorithm to the nonconvex Koopman operator learning problem for nonlinear dynamic systems. We show that the proposed algorithm will converge to a critical point with rate $O(1/T)$ and $O(\frac{1}{\log T})$ for the constant and diminishing learning rates, respectively, under some mild conditions. To cope with the high dimensional nonlinear dynamical systems, we present the first-ever distributed Koopman operator learning algorithm. We show that the distributed Koopman operator learning has the same convergence properties as the centralized Koopman operator learning, in the absence of optimal tracker, so long as the basis functions satisfy a set of state-based decomposition conditions. Numerical experiments are provided to complement our theoretical results. △ Less

Submitted 20 March, 2020; v1 submitted 30 September, 2019; originally announced September 2019.

Comments: 8 pages, 2 figures

Steady state programming of controlled nonlinear systems via deep dynamic mode decomposition

Authors: Aqib Hasnain, Nibodh Boddupalli, Shara Balakrishnan, Enoch Yeung

Abstract: This paper describes the optimal selection of a control policy to program the steady state of controlled nonlinear systems with hyperbolic fixed points. This work is motivated by the field of synthetic biology, in which saddle points are common (along with limit cycles), and the aim is to program cells to perform both digital and analog computation, though develo** genetic digital computation ha… ▽ More This paper describes the optimal selection of a control policy to program the steady state of controlled nonlinear systems with hyperbolic fixed points. This work is motivated by the field of synthetic biology, in which saddle points are common (along with limit cycles), and the aim is to program cells to perform both digital and analog computation, though develo** genetic digital computation has been the main focus. We frame the analog computing challenge of generating a steady state input-output function inside living cells. To program the steady state, a data-driven approach is taken wherein an approximation of the Koopman operator, identified via deep dynamic mode decomposition, is used to describe the dynamics of the system linearly. The new representation of the dynamics are then used to solve an optimization problem for the input which maximizes a direction in state space. Some added structure on the Koopman operator learning process for controlled systems is given for dynamics that are separable in the state and input. Finally, the methods are demonstrated on simulation examples of an incoherent feedforward loop and a combinatorial promoter system, two common network architectures seen in the field of synthetic biology. △ Less

Submitted 9 June, 2020; v1 submitted 29 September, 2019; originally announced September 2019.

Comments: 7 pages, 3 figures, accepted in 2020 American Control Conference

Information Transfer in Dynamical Systems and Optimal Placement of Actuators and Sensors for Control of Non-equilibrium Dynamics

Authors: Subhrajit Sinha, Umesh Vaidya, Enoch Yeung

Abstract: In this paper we develop the concept of information transfer between the Borel-measurable sets for a dynamical system described by a measurable space and a non-singular transformation. The concept is based on how Shannon entropy is transferred between the measurable sets, as the dynamical system evolves. We show that the proposed definition of information transfer satisfies the usual notions of in… ▽ More In this paper we develop the concept of information transfer between the Borel-measurable sets for a dynamical system described by a measurable space and a non-singular transformation. The concept is based on how Shannon entropy is transferred between the measurable sets, as the dynamical system evolves. We show that the proposed definition of information transfer satisfies the usual notions of information transfer and causality, namely, zero transfer and transfer asymmetry. Furthermore, we show how the information transfer measure can be used to classify ergodicity and mixing. We also develop the computational methods for information transfer computation and apply the framework for optimal placements of actuators and sensors for control of non-equilibrium dynamics. △ Less

Submitted 29 September, 2019; originally announced September 2019.

Online Learning of Dynamical Systems: An Operator Theoretic Approach

Authors: Subhrajit Sinha, Sai Pushpak Nandanoori, Enoch Yeung

Abstract: In this paper, we provide an algorithm for online computation of Koopman operator in real-time using streaming data. In recent years, there has been an increased interest in data-driven analysis of dynamical systems, with operator theoretic techniques being the most popular. Existing algorithms, like Dynamic Mode Decomposition (DMD) and Extended Dynamic Mode Decomposition (EDMD), use the entire da… ▽ More In this paper, we provide an algorithm for online computation of Koopman operator in real-time using streaming data. In recent years, there has been an increased interest in data-driven analysis of dynamical systems, with operator theoretic techniques being the most popular. Existing algorithms, like Dynamic Mode Decomposition (DMD) and Extended Dynamic Mode Decomposition (EDMD), use the entire data set for computation of the Koopman operator. However, many real life applications like power system analysis, biological systems, building systems etc. requires the real-time computation and updating of the Koopman operator, as new data streams in. In this paper, we propose an iterative algorithm for online computation of Koopman operator such that at each time step the Koopman operator is updated incrementally. In particular, we propose a Recursive Extended Dynamic Decomposition (rEDMD) algorithm for computation of Koopman operator from streaming data. Further, we test the algorithm in three different dynamical systems, namely, a linear system, a nonlinear system and a system governed by a Partial Differential Equation (PDE) and illustrate the computational efficiency of the iterative algorithm over the existing DMD and EDMD algorithms. △ Less

Submitted 27 September, 2019; originally announced September 2019.

A data-driven method for quantifying the impact of a genetic circuit on its host

Authors: Aqib Hasnain, Subhrajit Sinha, Yuval Dorfan, Amin Espah Borujeni, Yong** Park, Paul Maschhoff, Uma Saxena, Joshua Urrutia, Niall Gaffney, Diveena Becker, Atsede Siba, Narendra Maheshri, Ben Gordon, Chris Voigt, Enoch Yeung

Abstract: Genetic circuits are designed to implement certain logic in living cells, kee** burden on the host cell minimal. However, manipulating the genome often will have a significant impact for various reasons (usage of the cell machinery to express new genes, toxicity of genes, interactions with native genes, etc.). In this work we utilize Koopman operator theory to construct data-driven models of tra… ▽ More Genetic circuits are designed to implement certain logic in living cells, kee** burden on the host cell minimal. However, manipulating the genome often will have a significant impact for various reasons (usage of the cell machinery to express new genes, toxicity of genes, interactions with native genes, etc.). In this work we utilize Koopman operator theory to construct data-driven models of transcriptomic-level dynamics from noisy and temporally sparse RNAseq measurements. We show how Koopman models can be used to quantify impact on genetic circuits. We consider an experimental example, using high-throughput RNAseq measurements collected from wild-type E. coli, single gate components transformed in E. coli, and a NAND circuit composed from individual gates in E. coli, to explore how Koopman subspace functions encode increasing circuit interference on E. coli chassis dynamics. The algorithm provides a novel method for quantifying the impact of synthetic biological circuits on host-chassis dynamics. △ Less

Submitted 13 September, 2019; originally announced September 2019.

Comments: 4 pages Accepted in 2019 IEEE Biomedical Circuits and Systems Conference

Koopman Operators for Generalized Persistence of Excitation Conditions for Nonlinear Systems

Authors: Nibodh Boddupalli, Aqib Hasnain, Sai Pushpak Nandanoori, Enoch Yeung

Abstract: It is hard to identify nonlinear biological models strictly from data, with results that are often sensitive to experimental conditions. Automated experimental workflows and liquid handling enables unprecedented throughput, as well as the capacity to generate extremely large datasets. We seek to develop generalized identifiability conditions for informing the design of automated experiments to dis… ▽ More It is hard to identify nonlinear biological models strictly from data, with results that are often sensitive to experimental conditions. Automated experimental workflows and liquid handling enables unprecedented throughput, as well as the capacity to generate extremely large datasets. We seek to develop generalized identifiability conditions for informing the design of automated experiments to discover predictive nonlinear biological models. For linear systems, identifiability is characterized by persistence of excitation conditions. For nonlinear systems, no such persistence of excitation conditions exist. We use the input-Koopman operator method to model nonlinear systems and derive identifiability conditions for open-loop systems initialized from a single initial condition. We show that nonlinear identifiability is intrinsically tied to the rank of a given dataset's power spectral density, transformed through the lifted Koopman observable space. We illustrate these identifiability conditions with a simulated synthetic gene circuit model, the repressilator. We illustrate how rank degeneracy in datasets results in overfitted nonlinear models of the repressilator, resulting in poor predictive accuracy. Our findings provide novel experimental design criteria for discovery of globally predictive nonlinear models of biological phenomena. △ Less

Submitted 13 September, 2019; v1 submitted 24 June, 2019; originally announced June 2019.

Comments: 6 pages. 2 figures, accepted for 58th IEEE Conference on Decision and Control - Nice, France - December 11th-13th 2019

Optimal reporter placement in sparsely measured genetic networks using the Koopman operator

Authors: Aqib Hasnain, Nibodh Boddupalli, Enoch Yeung

Abstract: Optimal sensor placement is an important yet unsolved problem in control theory. In biological organisms, genetic activity is often highly nonlinear, making it difficult to design libraries of promoters to act as reporters of the cell state. We make use of the Koopman observability gramian to develop an algorithm for optimal sensor (or reporter) placement for discrete time nonlinear dynamical syst… ▽ More Optimal sensor placement is an important yet unsolved problem in control theory. In biological organisms, genetic activity is often highly nonlinear, making it difficult to design libraries of promoters to act as reporters of the cell state. We make use of the Koopman observability gramian to develop an algorithm for optimal sensor (or reporter) placement for discrete time nonlinear dynamical systems to ease the difficulty of design of the promoter library. This ease is enabled due to the fact that the Koopman operator represents the evolution of a nonlinear system linearly by lifting the states to an infinite-dimensional space of observables. The Koopman framework ideally demands high temporal resolution, but data in biology are often sampled sparsely in time. Therefore we compute what we call the temporally fine-grained Koopman operator from the temporally coarse-grained Koopman operator, the latter of which is identified from the sparse data. The optimal placement of sensors then corresponds to maximizing the observability of the fine-grained system. We demonstrate the algorithm on a simulation example of a circadian oscillator. △ Less

Submitted 18 September, 2019; v1 submitted 3 June, 2019; originally announced June 2019.

Comments: 6 pages, 3 figures, to appear in 2019 IEEE Conference on Decision and Control (CDC)

On Computation of Koopman Operator from Sparse Data

Authors: Subhrajit Sinha, Enoch Yeung

Abstract: In this paper we propose a novel approach to compute the Koopman operator from sparse time series data. In recent years there has been considerable interests in operator theoretic methods for data-driven analysis of dynamical systems. Existing techniques for the approximation of the Koopman operator require sufficiently large data sets, but in many applications, the data set may not be large enoug… ▽ More In this paper we propose a novel approach to compute the Koopman operator from sparse time series data. In recent years there has been considerable interests in operator theoretic methods for data-driven analysis of dynamical systems. Existing techniques for the approximation of the Koopman operator require sufficiently large data sets, but in many applications, the data set may not be large enough to approximate the operators to acceptable limits. In this paper, using ideas from robust optimization, we propose an algorithm to compute the Koopman operator from sparse data. We enrich the sparse data set with artificial data points, generated by adding bounded artificial noise and and formulate the noisy robust learning problem as a robust optimization problem and show that the optimal solution is the Koopman operator with smallest error. We illustrate the efficiency of our proposed approach in three different dynamical systems, namely, a linear system, a nonlinear system and a dynamical system governed by a partial differential equation. △ Less

Submitted 10 January, 2019; originally announced January 2019.

A Constructive Approach for One-Shot Training of Neural Networks Using Hypercube-Based Topological Coverings

Authors: W. Brent Daniel, Enoch Yeung

Abstract: In this paper we presented a novel constructive approach for training deep neural networks using geometric approaches. We show that a topological covering can be used to define a class of distributed linear matrix inequalities, which in turn directly specify the shape and depth of a neural network architecture. The key insight is a fundamental relationship between linear matrix inequalities and th… ▽ More In this paper we presented a novel constructive approach for training deep neural networks using geometric approaches. We show that a topological covering can be used to define a class of distributed linear matrix inequalities, which in turn directly specify the shape and depth of a neural network architecture. The key insight is a fundamental relationship between linear matrix inequalities and their ability to bound the shape of data, and the rectified linear unit (ReLU) activation function employed in modern neural networks. We show that unit cover geometry and cover porosity are two design variables in cover-constructive learning that play a critical role in defining the complexity of the model and generalizability of the resulting neural network classifier. In the context of cover-constructive learning, these findings underscore the age old trade-off between model complexity and overfitting (as quantified by the number of elements in the data cover) and generalizability on test data. Finally, we benchmark on algorithm on the Iris, MNIST, and Wine dataset and show that the constructive algorithm is able to train a deep neural network classifier in one shot, achieving equal or superior levels of training and test classification accuracy with reduced training time. △ Less

Submitted 9 January, 2019; originally announced January 2019.

Assessing the distribution of discrete survival time in presence of recall error

Authors: Sedigheh Mirzaei Salehabadi, Edwina Yeung, Germaine M. Buck Louis, Rajeshwari Sundaram

Abstract: Retrospectively ascertained survival time may be subject to recall error. An example of discrete survival time with such recall error is time-to-pregnancy (TTP), the number of months non-contracepting couples require to get pregnant which is a measure of human fecundity. The epidemiological literature has demonstrated that retrospective TTP is subject to recall error and statistical models focusin… ▽ More Retrospectively ascertained survival time may be subject to recall error. An example of discrete survival time with such recall error is time-to-pregnancy (TTP), the number of months non-contracepting couples require to get pregnant which is a measure of human fecundity. The epidemiological literature has demonstrated that retrospective TTP is subject to recall error and statistical models focusing on TTP have not accounted for the recall error. We propose a multistage model that utilizes women's retrospectively-reported TTP and associated certainty to estimate the TTP distribution. Our proposed model utilizes a discrete survival function that accounts for random heterogeneity arising from between women TTP data as well as a multinomial regression model to account for her certainty as accuracy may decline over time, i.e., depends on time since pregnancy in estimating the TTP distribution. Other novel features of the model include attention to whether the pregnancy was (un)planned as well as providing an approach to predict survival function for women without a reported TTP. Our model allows for the consideration of covariates for each of the underlying factors of (un)planned pregnancy, measure of certainty and TTP distribution. The proposed model is applicable for any discrete survival time when certainty in reporting may be a consideration. We use Monte Carlo simulations to assess the finite sample performance for the proposed estimators. We illustrate our proposed method using data from Upstate KIDS Study. △ Less

Submitted 16 October, 2018; originally announced October 2018.

Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks

Authors: Panos Stinis, Tobias Hagge, Alexandre M. Tartakovsky, Enoch Yeung

Abstract: We suggest ways to enforce given constraints in the output of a Generative Adversarial Network (GAN) generator both for interpolation and extrapolation (prediction). For the case of dynamical systems, given a time series, we wish to train GAN generators that can be used to predict trajectories starting from a given initial condition. In this setting, the constraints can be in algebraic and/or diff… ▽ More We suggest ways to enforce given constraints in the output of a Generative Adversarial Network (GAN) generator both for interpolation and extrapolation (prediction). For the case of dynamical systems, given a time series, we wish to train GAN generators that can be used to predict trajectories starting from a given initial condition. In this setting, the constraints can be in algebraic and/or differential form. Even though we are predominantly interested in the case of extrapolation, we will see that the tasks of interpolation and extrapolation are related. However, they need to be treated differently. For the case of interpolation, the incorporation of constraints is built into the training of the GAN. The incorporation of the constraints respects the primary game-theoretic setup of a GAN so it can be combined with existing algorithms. However, it can exacerbate the problem of instability during training that is well-known for GANs. We suggest adding small noise to the constraints as a simple remedy that has performed well in our numerical experiments. The case of extrapolation (prediction) is more involved. During training, the GAN generator learns to interpolate a noisy version of the data and we enforce the constraints. This approach has connections with model reduction that we can utilize to improve the efficiency and accuracy of the training. Depending on the form of the constraints, we may enforce them also during prediction through a projection step. We provide examples of linear and nonlinear systems of differential equations to illustrate the various constructions. △ Less

Submitted 19 June, 2019; v1 submitted 21 March, 2018; originally announced March 2018.

Comments: 29 pages; v2 has major text revision/restructuring, includes results for the Lorenz system and has several more references

Report number: PNNL-SA-133233 MSC Class: 68T05; 65L05; 37M10; 62M45; 68Q32

Decomposition of Nonlinear Dynamical Networks via Comparison Systems

Authors: Abdullah Maruf, Soumya Kundu, Enoch Yeung, Marian Anghel

Abstract: In analysis and control of large-scale nonlinear dynamical systems, a distributed approach is often an attractive option due to its computational tractability and usually low communication requirements. Success of the distributed control design relies on the separability of the network into weakly interacting subsystems such that minimal information exchange between subsystems is sufficient to ach… ▽ More In analysis and control of large-scale nonlinear dynamical systems, a distributed approach is often an attractive option due to its computational tractability and usually low communication requirements. Success of the distributed control design relies on the separability of the network into weakly interacting subsystems such that minimal information exchange between subsystems is sufficient to achieve satisfactory control performance. While distributed analysis and control design for dynamical network have been well studied, decomposition of nonlinear networks into weakly interacting subsystems has not received as much attention. In this article we propose a vector Lyapunov functions based approach to quantify the energy-flow in a dynamical network via a model of a comparison system. Introducing a notion of power and energy flow in a dynamical network, we use sum-of-squares programming tools to partition polynomial networks into weakly interacting subsystems. Examples are provided to illustrate the proposed method of decomposition. △ Less

Submitted 8 March, 2018; originally announced March 2018.

Comments: to be presented at ECC 2018

Report number: PNNL-SA-130627

A Class of Logistic Functions for Approximating State-Inclusive Koopman Operators

Authors: Charles A. Johnson, Enoch Yeung

Abstract: An outstanding challenge in nonlinear systems theory is identification or learning of a given nonlinear system's Koopman operator directly from data or models. Advances in extended dynamic mode decomposition approaches and machine learning methods have enabled data-driven discovery of Koopman operators, for both continuous and discrete-time systems. Since Koopman operators are often infinite-dimen… ▽ More An outstanding challenge in nonlinear systems theory is identification or learning of a given nonlinear system's Koopman operator directly from data or models. Advances in extended dynamic mode decomposition approaches and machine learning methods have enabled data-driven discovery of Koopman operators, for both continuous and discrete-time systems. Since Koopman operators are often infinite-dimensional, they are approximated in practice using finite-dimensional systems. The fidelity and convergence of a given finite-dimensional Koopman approximation is a subject of ongoing research. In this paper we introduce a class of Koopman observable functions that confer an approximate closure property on their corresponding finite-dimensional approximations of the Koopman operator. We derive error bounds for the fidelity of this class of observable functions, as well as identify two key learning parameters which can be used to tune performance. We illustrate our approach on two classical nonlinear system models: the Van Der Pol oscillator and the bistable toggle switch. △ Less

Submitted 8 December, 2017; originally announced December 2017.

Comments: 8 pages

Solving differential equations with unknown constitutive relations as recurrent neural networks

Authors: Tobias Hagge, Panos Stinis, Enoch Yeung, Alexandre M. Tartakovsky

Abstract: We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and we use recurrent neural network to "learn" the reaction rate from this data. This is achieved by including a discretized ordinary differential equations as part of a recurrent neural networ… ▽ More We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and we use recurrent neural network to "learn" the reaction rate from this data. This is achieved by including a discretized ordinary differential equations as part of a recurrent neural network training problem. We extend TensorFlow's recurrent neural network architecture to create a simple but scalable and effective solver for the unknown functions, and apply it to a fedbatch bioreactor simulation problem. Use of techniques from recent deep learning literature enables training of functions with behavior manifesting over thousands of time steps. Our networks are structurally similar to recurrent neural networks, but differences in design and function require modifications to the conventional wisdom about training such networks. △ Less

Submitted 5 October, 2017; originally announced October 2017.

Comments: 19 pages, 8 figures

Decomposition of Nonlinear Dynamical Systems Using Koopman Gramians

Authors: Zhiyuan Liu, Soumya Kundu, Lijun Chen, Enoch Yeung

Abstract: In this paper we propose a new Koopman operator approach to the decomposition of nonlinear dynamical systems using Koopman Gramians. We introduce the notion of an input-Koopman operator, and show how input-Koopman operators can be used to cast a nonlinear system into the classical state-space form, and identify conditions under which input and state observable functions are well separated. We then… ▽ More In this paper we propose a new Koopman operator approach to the decomposition of nonlinear dynamical systems using Koopman Gramians. We introduce the notion of an input-Koopman operator, and show how input-Koopman operators can be used to cast a nonlinear system into the classical state-space form, and identify conditions under which input and state observable functions are well separated. We then extend an existing method of dynamic mode decomposition for learning Koopman operators from data known as deep dynamic mode decomposition to systems with controls or disturbances. We illustrate the accuracy of the method in learning an input-state separable Koopman operator for an example system, even when the underlying system exhibits mixed state-input terms. We next introduce a nonlinear decomposition algorithm, based on Koopman Gramians, that maximizes internal subsystem observability and disturbance rejection from unwanted noise from other subsystems. We derive a relaxation based on Koopman Gramians and multi-way partitioning for the resulting NP-hard decomposition problem. We lastly illustrate the proposed algorithm with the swing dynamics for an IEEE 39-bus system. △ Less

Submitted 4 October, 2017; originally announced October 2017.

Comments: 8 pages, submitted to IEEE 2018 ACC

A Koopman Operator Approach for Computing and Balancing Gramians for Discrete Time Nonlinear Systems

Authors: Enoch Yeung, Zhiyuan Liu, Nathan O. Hodas

Abstract: In this paper, we consider the problem of quantifying controllability and observability of a nonlinear discrete time dynamical system. We introduce the Koopman operator as a canonical representation of the system and apply a lifting technique to compute gramians in the space of full-state observables. We illustrate the properties of these gramians and identify several relationships with canonical… ▽ More In this paper, we consider the problem of quantifying controllability and observability of a nonlinear discrete time dynamical system. We introduce the Koopman operator as a canonical representation of the system and apply a lifting technique to compute gramians in the space of full-state observables. We illustrate the properties of these gramians and identify several relationships with canonical results on local controllability and observability. Once defined, we show that these gramians can be balanced through a change of coordinates on the observables space, which in turn allows for direct application of balanced truncation. Throughout the paper, we highlight the aspects of our approach with an example nonlinear system. △ Less

Submitted 25 September, 2017; originally announced September 2017.

Comments: 9 pages, 4 figures, IEEE conference format

Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems

Authors: Enoch Yeung, Soumya Kundu, Nathan Hodas

Abstract: The Koopman operator has recently garnered much attention for its value in dynamical systems analysis and data-driven model discovery. However, its application has been hindered by the computational complexity of extended dynamic mode decomposition; this requires a combinatorially large basis set to adequately describe many nonlinear systems of interest, e.g. cyber-physical infrastructure systems,… ▽ More The Koopman operator has recently garnered much attention for its value in dynamical systems analysis and data-driven model discovery. However, its application has been hindered by the computational complexity of extended dynamic mode decomposition; this requires a combinatorially large basis set to adequately describe many nonlinear systems of interest, e.g. cyber-physical infrastructure systems, biological networks, social systems, and fluid dynamics. Often the dictionaries generated for these problems are manually curated, requiring domain-specific knowledge and painstaking tuning. In this paper we introduce a deep learning framework for learning Koopman operators of nonlinear dynamical systems. We show that this novel method automatically selects efficient deep dictionaries, outperforming state-of-the-art methods. We benchmark this method on partially observed nonlinear systems, including the glycolytic oscillator and show it is able to predict quantitatively 100 steps into the future, using only a single timepoint, and qualitative oscillatory behavior 400 steps into the future. △ Less

Submitted 17 November, 2017; v1 submitted 22 August, 2017; originally announced August 2017.

Comments: 16 pages, 5 figures

Capacity Analysis of Linear Operator Channels over Finite Fields

Authors: Shenghao Yang, Siu-Wai Ho, ** Meng, En-hui Yeung

Abstract: Motivated by communication through a network employing linear network coding, capacities of linear operator channels (LOCs) with arbitrarily distributed transfer matrices over finite fields are studied. Both the Shannon capacity $C$ and the subspace coding capacity $C_{\text{SS}}$ are analyzed. By establishing and comparing lower bounds on $C$ and upper bounds on $C_{\text{SS}}$, various necessary… ▽ More Motivated by communication through a network employing linear network coding, capacities of linear operator channels (LOCs) with arbitrarily distributed transfer matrices over finite fields are studied. Both the Shannon capacity $C$ and the subspace coding capacity $C_{\text{SS}}$ are analyzed. By establishing and comparing lower bounds on $C$ and upper bounds on $C_{\text{SS}}$, various necessary conditions and sufficient conditions such that $C=C_{\text{SS}}$ are obtained. A new class of LOCs such that $C=C_{\text{SS}}$ is identified, which includes LOCs with uniform-given-rank transfer matrices as special cases. It is also demonstrated that $C_{\text{SS}}$ is strictly less than $C$ for a broad class of LOCs. In general, an optimal subspace coding scheme is difficult to find because it requires to solve the maximization of a non-concave function. However, for a LOC with a unique subspace degradation, $C_{\text{SS}}$ can be obtained by solving a convex optimization problem over rank distribution. Classes of LOCs with a unique subspace degradation are characterized. Since LOCs with uniform-given-rank transfer matrices have unique subspace degradations, some existing results on LOCs with uniform-given-rank transfer matrices are explained from a more general way. △ Less

Submitted 26 February, 2014; v1 submitted 22 August, 2011; originally announced August 2011.

Comments: To appear in IEEE Transactions on Information Theory

Journal ref: Information Theory, IEEE Transactions on , vol.60, no.8, pp.4880-4901, Aug. 2014

The Meaning of Structure in Interconnected Dynamic Systems

Authors: E. Yeung, J. Goncalves, H. Sandberg, S. Warnick

Abstract: Interconnected dynamic systems are a pervasive component of our modern infrastructures. The complexity of such systems can be staggering, which motivates simplified representations for their manipulation and analysis. This work introduces the complete computational structure of a system as a common baseline for comparing different simplified representations. Linear systems are then used as a vehic… ▽ More Interconnected dynamic systems are a pervasive component of our modern infrastructures. The complexity of such systems can be staggering, which motivates simplified representations for their manipulation and analysis. This work introduces the complete computational structure of a system as a common baseline for comparing different simplified representations. Linear systems are then used as a vehicle for comparing and contrasting distinct partial structure representations. Such representations simplify the description of a system's complete computational structure at various levels of fidelity while retaining a full description of the system's input-output dynamic behavior. Relationships between these various partial structure representations are detailed, and the landscape of new realization, minimality, and model reduction problems introduced by these representations is briefly surveyed. △ Less

Submitted 12 August, 2011; originally announced August 2011.

Comments: 21 pages, to appear in IEEE Control Systems Magazine 2012 Special Invited Issue: Designing Controls for Modern Infrastructure Networks