Showing 1–50 of 70 results for author: Magri, L

Quantum Computing for nonlinear differential equations and turbulence

Authors: Felix Tennie, Sylvain Laizet, Seth Lloyd, Luca Magri

Abstract: A large spectrum of problems in classical physics and engineering, such as turbulence, is governed by nonlinear differential equations, which typically require high-performance computing to be solved. Over the past decade, however, the growth of classical computing power has slowed down because the miniaturisation of chips has been approaching the atomic scale. This is marking an end to Moore's la… ▽ More A large spectrum of problems in classical physics and engineering, such as turbulence, is governed by nonlinear differential equations, which typically require high-performance computing to be solved. Over the past decade, however, the growth of classical computing power has slowed down because the miniaturisation of chips has been approaching the atomic scale. This is marking an end to Moore's law, which calls for a new computing paradigm: Quantum computing is a prime candidate. In this paper, we offer a perspective on the current challenges that need to be overcome in order to use quantum computing for the simulation of nonlinear dynamics. We review and discuss progress in the development of both quantum algorithms for nonlinear equations and quantum hardware. We propose pairings between quantum algorithms for nonlinear equations and quantum hardware concepts. These avenues open new opportunities for the simulation of nonlinear systems and turbulence. △ Less

Submitted 7 June, 2024; originally announced June 2024.

An expert-driven data generation pipeline for histological images

Authors: Roberto Basla, Loris Giulivi, Luca Magri, Giacomo Boracchi

Abstract: Deep Learning (DL) models have been successfully applied to many applications including biomedical cell segmentation and classification in histological images. These models require large amounts of annotated data which might not always be available, especially in the medical field where annotations are scarce and expensive. To overcome this limitation, we propose a novel pipeline for generating sy… ▽ More Deep Learning (DL) models have been successfully applied to many applications including biomedical cell segmentation and classification in histological images. These models require large amounts of annotated data which might not always be available, especially in the medical field where annotations are scarce and expensive. To overcome this limitation, we propose a novel pipeline for generating synthetic datasets for cell segmentation. Given only a handful of annotated images, our method generates a large dataset of images which can be used to effectively train DL instance segmentation models. Our solution is designed to generate cells of realistic shapes and placement by allowing experts to incorporate domain knowledge during the generation of the dataset. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: 5 pages, Accepted at the International Symposium on Biomedical Imaging (ISBI) 2024, Code available at https://github.com/rb-sl/ExpertDrivenNuclei

MSC Class: 68T45 ACM Class: I.2.10

Minimal Perspective Autocalibration

Authors: Andrea Porfiri Dal Cin, Timothy Duff, Luca Magri, Tomas Pajdla

Abstract: We introduce a new family of minimal problems for reconstruction from multiple views. Our primary focus is a novel approach to autocalibration, a long-standing problem in computer vision. Traditional approaches to this problem, such as those based on Kruppa's equations or the modulus constraint, rely explicitly on the knowledge of multiple fundamental matrices or a projective reconstruction. In co… ▽ More We introduce a new family of minimal problems for reconstruction from multiple views. Our primary focus is a novel approach to autocalibration, a long-standing problem in computer vision. Traditional approaches to this problem, such as those based on Kruppa's equations or the modulus constraint, rely explicitly on the knowledge of multiple fundamental matrices or a projective reconstruction. In contrast, we consider a novel formulation involving constraints on image points, the unknown depths of 3D points, and a partially specified calibration matrix $K$. For $2$ and $3$ views, we present a comprehensive taxonomy of minimal autocalibration problems obtained by relaxing some of these constraints. These problems are organized into classes according to the number of views and any assumed prior knowledge of $K$. Within each class, we determine problems with the fewest -- or a relatively small number of -- solutions. From this zoo of problems, we devise three practical solvers. Experiments with synthetic and real data and interfacing our solvers with COLMAP demonstrate that we achieve superior accuracy compared to state-of-the-art calibration methods. The code is available at https://github.com/andreadalcin/MinimalPerspectiveAutocalibration △ Less

Submitted 9 May, 2024; originally announced May 2024.

Comments: 8 pages main paper + 2 pages references + 8 pages supplementary; to be presented at CVPR 2024

Prediction of chaotic dynamics and extreme events: A recurrence-free quantum reservoir computing approach

Authors: Osama Ahmed, Felix Tennie, Luca Magri

Abstract: In chaotic dynamical systems, extreme events manifest in time series as unpredictable large-amplitude peaks. Although deterministic, extreme events appear seemingly randomly, which makes their forecasting difficult. By learning the dynamics from observables (data), reservoir computers can time-accurately predict extreme events and chaotic dynamics, but they may require many degrees of freedom (lar… ▽ More In chaotic dynamical systems, extreme events manifest in time series as unpredictable large-amplitude peaks. Although deterministic, extreme events appear seemingly randomly, which makes their forecasting difficult. By learning the dynamics from observables (data), reservoir computers can time-accurately predict extreme events and chaotic dynamics, but they may require many degrees of freedom (large reservoirs). In this paper, by exploiting quantum-computer ansätze and entanglement, we design reservoir computers with compact reservoirs and accurate prediction capabilities. First, we propose the recurrence-free quantum reservoir computer (RF-QRC) architecture. By develo** ad-hoc quantum feature maps and removing recurrent connections, the RF-QRC has quantum circuits with small depths. This allows the RF-QRC to scale well with higher-dimensional chaotic systems, which makes it suitable for hardware implementation. Second, we forecast the temporal chaotic dynamics and their long-term statistics of low- and higher-dimensional dynamical systems. We find that RF-QRC requires smaller reservoirs than classical reservoir computers. Third, we apply the RF-QRC to the time prediction of extreme events in a model of a turbulent shear flow with turbulent bursts. We find that the RF-QRC has a longer predictability than the classical reservoir computer. The results and analyses indicate that quantum-computer ansätze offer nonlinear expressivity and computational scalability, which are useful for forecasting chaotic dynamics and extreme events. This work opens new opportunities for using quantum machine learning on near-term quantum computers. △ Less

Submitted 6 May, 2024; originally announced May 2024.

Decoder Decomposition for the Analysis of the Latent Space of Nonlinear Autoencoders With Wind-Tunnel Experimental Data

Authors: Yaxin Mo, Tullio Traverso, Luca Magri

Abstract: Turbulent flows are chaotic and multi-scale dynamical systems, which have large numbers of degrees of freedom. Turbulent flows, however, can be modelled with a smaller number of degrees of freedom when using the appropriate coordinate system, which is the goal of dimensionality reduction via nonlinear autoencoders. Autoencoders are expressive tools, but they are difficult to interpret. The goal of… ▽ More Turbulent flows are chaotic and multi-scale dynamical systems, which have large numbers of degrees of freedom. Turbulent flows, however, can be modelled with a smaller number of degrees of freedom when using the appropriate coordinate system, which is the goal of dimensionality reduction via nonlinear autoencoders. Autoencoders are expressive tools, but they are difficult to interpret. The goal of this paper is to propose a method to aid the interpretability of autoencoders. This is the decoder decomposition. First, we propose the decoder decomposition, which is a post-processing method to connect the latent variables to the coherent structures of flows. Second, we apply the decoder decomposition to analyse the latent space of synthetic data of a two-dimensional unsteady wake past a cylinder. We find that the dimension of latent space has a significant impact on the interpretability of autoencoders. We identify the physical and spurious latent variables. Third, we apply the decoder decomposition to the latent space of wind-tunnel experimental data of a three-dimensional turbulent wake past a bluff body. We show that the reconstruction error is a function of both the latent space dimension and the decoder size, which are correlated. Finally, we apply the decoder decomposition to rank and select latent variables based on the coherent structures that they represent. This is useful to filter unwanted or spurious latent variables, or to pinpoint specific coherent structures of interest. The ability to rank and select latent variables will help users design and interpret nonlinear autoencoders. △ Less

Submitted 25 April, 2024; originally announced April 2024.

A real-time digital twin of azimuthal thermoacoustic instabilities

Authors: Andrea Nóvoa, Nicolas Noiray, James R. Dawson, Luca Magri

Abstract: When they occur, azimuthal thermoacoustic oscillations can detrimentally affect the safe operation of gas turbines and aeroengines. We develop a real-time digital twin of azimuthal thermoacoustics of a hydrogen-based annular combustor. The digital twin seamlessly combines two sources of information about the system (i) a physics-based low-order model; and (ii) raw and sparse experimental data from… ▽ More When they occur, azimuthal thermoacoustic oscillations can detrimentally affect the safe operation of gas turbines and aeroengines. We develop a real-time digital twin of azimuthal thermoacoustics of a hydrogen-based annular combustor. The digital twin seamlessly combines two sources of information about the system (i) a physics-based low-order model; and (ii) raw and sparse experimental data from microphones, which contain both aleatoric noise and turbulent fluctuations. First, we derive a low-order thermoacoustic model for azimuthal instabilities, which is deterministic. Second, we propose a real-time data assimilation framework to infer the acoustic pressure, the physical parameters, and the model and measurement biases simultaneously. This is the bias-regularized ensemble Kalman filter (r-EnKF), for which we find an analytical solution that solves the optimization problem. Third, we propose a reservoir computer, which infers both the model bias and measurement bias to close the assimilation equations. Fourth, we propose a real-time digital twin of the azimuthal thermoacoustic dynamics of a laboratory hydrogen-based annular combustor for a variety of equivalence ratios. We find that the real-time digital twin (i) autonomously predicts azimuthal dynamics, in contrast to bias-unregularized methods; (ii) uncovers the physical acoustic pressure from the raw data, i.e., it acts as a physics-based filter; (iii) is a time-varying parameter system, which generalizes existing models that have constant parameters, and capture only slow-varying variables. The digital twin generalizes to all equivalence ratios, which bridges the gap of existing models. This work opens new opportunities for real-time digital twinning of multi-physics problems. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: 23 pages

Adjoint Sensitivities of Chaotic Flows without Adjoint Solvers: A Data-Driven Approach

Authors: Defne E. Ozan, Luca Magri

Abstract: In one calculation, adjoint sensitivity analysis provides the gradient of a quantity of interest with respect to all system's parameters. Conventionally, adjoint solvers need to be implemented by differentiating computational models, which can be a cumbersome task and is code-specific. To propose an adjoint solver that is not code-specific, we develop a data-driven strategy. We demonstrate its app… ▽ More In one calculation, adjoint sensitivity analysis provides the gradient of a quantity of interest with respect to all system's parameters. Conventionally, adjoint solvers need to be implemented by differentiating computational models, which can be a cumbersome task and is code-specific. To propose an adjoint solver that is not code-specific, we develop a data-driven strategy. We demonstrate its application on the computation of gradients of long-time averages of chaotic flows. First, we deploy a parameter-aware echo state network (ESN) to accurately forecast and simulate the dynamics of a dynamical system for a range of system's parameters. Second, we derive the adjoint of the parameter-aware ESN. Finally, we combine the parameter-aware ESN with its adjoint version to compute the sensitivities to the system parameters. We showcase the method on a prototypical chaotic system. Because adjoint sensitivities in chaotic regimes diverge for long integration times, we analyse the application of ensemble adjoint method to the ESN. We find that the adjoint sensitivities obtained from the ESN match closely with the original system. This work opens possibilities for sensitivity analysis without code-specific adjoint solvers. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Data-driven computation of adjoint sensitivities without adjoint solvers: An application to thermoacoustics

Authors: Defne E. Ozan, Luca Magri

Abstract: Adjoint methods have been the pillar of gradient-based optimization for decades. They enable the accurate computation of a gradient (sensitivity) of a quantity of interest with respect to all system's parameters in one calculation. When the gradient is embedded in an optimization routine, the quantity of interest can be optimized for the system to have the desired behaviour. Adjoint methods requir… ▽ More Adjoint methods have been the pillar of gradient-based optimization for decades. They enable the accurate computation of a gradient (sensitivity) of a quantity of interest with respect to all system's parameters in one calculation. When the gradient is embedded in an optimization routine, the quantity of interest can be optimized for the system to have the desired behaviour. Adjoint methods require the system's Jacobian, whose computation can be cumbersome, and is problem dependent. We propose a computational strategy to infer the adjoint sensitivities from data (observables), which bypasses the need of the Jacobian of the physical system. The key component of this strategy is an echo state network, which learns the dynamics of nonlinear regimes with varying parameters, and evolves dynamically via a hidden state. Although the framework is general, we focus on thermoacoustics governed by nonlinear and time-delayed systems. First, we show that a parameter-aware Echo State Network (ESN) infers the parameterized dynamics. Second, we derive the adjoint of the ESN to compute the sensitivity of time-averaged cost functionals. Third, we propose the Thermoacoustic Echo State Network (T-ESN), which hard constrains the physical knowledge in the network architecture. Fourth, we apply the framework to a variety of nonlinear thermoacoustic regimes of a prototypical system. We show that the T-ESN accurately infers the correct adjoint sensitivities of the time-averaged acoustic energy with respect to the flame parameters. The results are robust to noisy data, from periodic, through quasiperiodic, to chaotic regimes. A single network predicts the nonlinear bifurcations on unseen scenarios, and so the inferred adjoint sensitivities are employed to suppress an instability via steepest descent. This work opens new possibilities for gradient-based data-driven design optimization. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Computing distances and means on manifolds with a metric-constrained Eikonal approach

Authors: Daniel Kelshaw, Luca Magri

Abstract: Computing distances on Riemannian manifolds is a challenging problem with numerous applications, from physics, through statistics, to machine learning. In this paper, we introduce the metric-constrained Eikonal solver to obtain continuous, differentiable representations of distance functions on manifolds. The differentiable nature of these representations allows for the direct computation of globa… ▽ More Computing distances on Riemannian manifolds is a challenging problem with numerous applications, from physics, through statistics, to machine learning. In this paper, we introduce the metric-constrained Eikonal solver to obtain continuous, differentiable representations of distance functions on manifolds. The differentiable nature of these representations allows for the direct computation of globally length-minimising paths on the manifold. We showcase the use of metric-constrained Eikonal solvers for a range of manifolds and demonstrate the applications. First, we demonstrate that metric-constrained Eikonal solvers can be used to obtain the Fréchet mean on a manifold, employing the definition of a Gaussian mixture model, which has an analytical solution to verify the numerical results. Second, we demonstrate how the obtained distance function can be used to conduct unsupervised clustering on the manifold -- a task for which existing approaches are computationally prohibitive. This work opens opportunities for distance computations on manifolds. △ Less

Submitted 12 April, 2024; originally announced April 2024.

Solving nonlinear differential equations on Quantum Computers: A Fokker-Planck approach

Authors: Felix Tennie, Luca Magri

Abstract: For quantum computers to become useful tools to physicists, engineers and computational scientists, quantum algorithms for solving nonlinear differential equations need to be developed. Despite recent advances, the quest for a solver that can integrate nonlinear dynamical systems with a quantum advantage, whilst being realisable on available (or near-term) quantum hardware, is an open challenge. I… ▽ More For quantum computers to become useful tools to physicists, engineers and computational scientists, quantum algorithms for solving nonlinear differential equations need to be developed. Despite recent advances, the quest for a solver that can integrate nonlinear dynamical systems with a quantum advantage, whilst being realisable on available (or near-term) quantum hardware, is an open challenge. In this paper, we propose to transform a nonlinear dynamical system into a linear system, which we integrate with quantum algorithms. Key to the method is the Fokker-Planck equation, which is a non-normal partial differential equation. Three integration strategies are proposed: (i) Forward-Euler step** by unitary block encoding; (ii) Schroedingerisation, and (iii) Forward-Euler step** by linear addition of unitaries. We emulate the integration of prototypical nonlinear systems with the proposed quantum solvers, and compare the output with the benchmark solutions of classical integrators. We find that classical and quantum outputs are in good agreement. This paper opens opportunities for solving nonlinear differential equations with quantum algorithms. △ Less

Submitted 24 January, 2024; originally announced January 2024.

Physics-constrained convolutional neural networks for inverse problems in spatiotemporal partial differential equations

Authors: Daniel Kelshaw, Luca Magri

Abstract: We propose a physics-constrained convolutional neural network (PC-CNN) to solve two types of inverse problems in partial differential equations (PDEs), which are nonlinear and vary both in space and time. In the first inverse problem, we are given data that is offset by spatially varying systematic error (i.e., the bias, also known as the epistemic uncertainty). The task is to uncover from the bia… ▽ More We propose a physics-constrained convolutional neural network (PC-CNN) to solve two types of inverse problems in partial differential equations (PDEs), which are nonlinear and vary both in space and time. In the first inverse problem, we are given data that is offset by spatially varying systematic error (i.e., the bias, also known as the epistemic uncertainty). The task is to uncover from the biased data the true state, which is the solution of the PDE. In the second inverse problem, we are given sparse information on the solution of a PDE. The task is to reconstruct the solution in space with high-resolution. First, we present the PC-CNN, which constrains the PDE with a simple time-windowing scheme to handle sequential data. Second, we analyse the performance of the PC-CNN for uncovering solutions from biased data. We analyse both linear and nonlinear convection-diffusion equations, and the Navier-Stokes equations, which govern the spatiotemporally chaotic dynamics of turbulent flows. We find that the PC-CNN correctly recovers the true solution for a variety of biases, which are parameterised as non-convex functions. Third, we analyse the performance of the PC-CNN for reconstructing solutions from biased data for the turbulent flow. We reconstruct the spatiotemporal chaotic solution on a high-resolution grid from only 2\% of the information contained in it. For both tasks, we further analyse the Navier-Stokes solutions. We find that the inferred solutions have a physical spectral energy content, whereas traditional methods, such as interpolation, do not. This work opens opportunities for solving inverse problems with partial differential equations. △ Less

Submitted 1 February, 2024; v1 submitted 18 January, 2024; originally announced January 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2306.04600, arXiv:2306.10990

Manifold-augmented Eikonal Equations: Geodesic Distances and Flows on Differentiable Manifolds

Authors: Daniel Kelshaw, Luca Magri

Abstract: Manifolds discovered by machine learning models provide a compact representation of the underlying data. Geodesics on these manifolds define locally length-minimising curves and provide a notion of distance, which are key for reduced-order modelling, statistical inference, and interpolation. In this work, we propose a model-based parameterisation for distance fields and geodesic flows on manifolds… ▽ More Manifolds discovered by machine learning models provide a compact representation of the underlying data. Geodesics on these manifolds define locally length-minimising curves and provide a notion of distance, which are key for reduced-order modelling, statistical inference, and interpolation. In this work, we propose a model-based parameterisation for distance fields and geodesic flows on manifolds, exploiting solutions of a manifold-augmented Eikonal equation. We demonstrate how the geometry of the manifold impacts the distance field, and exploit the geodesic flow to obtain globally length-minimising curves directly. This work opens opportunities for statistics and reduced-order modelling on differentiable manifolds. △ Less

Submitted 2 November, 2023; v1 submitted 9 October, 2023; originally announced October 2023.

Comments: Accepted to NeurIPS 2023: Symmetry and Geometry in Neural Representations Workshop

Control-aware echo state networks (Ca-ESN) for the suppression of extreme events

Authors: Alberto Racca, Luca Magri

Abstract: Extreme event are sudden large-amplitude changes in the state or observables of chaotic nonlinear systems, which characterize many scientific phenomena. Because of their violent nature, extreme events typically have adverse consequences, which call for methods to prevent the events from happening. In this work, we introduce the control-aware echo state network (Ca-ESN) to seamlessly combine ESNs a… ▽ More Extreme event are sudden large-amplitude changes in the state or observables of chaotic nonlinear systems, which characterize many scientific phenomena. Because of their violent nature, extreme events typically have adverse consequences, which call for methods to prevent the events from happening. In this work, we introduce the control-aware echo state network (Ca-ESN) to seamlessly combine ESNs and control strategies, such as proportional-integral-derivative and model predictive control, to suppress extreme events. The methodology is showcased on a chaotic-turbulent flow, in which we reduce the occurrence of extreme events with respect to traditional methods by two orders of magnitude. This works opens up new possibilities for the efficient control of nonlinear systems with neural networks. △ Less

Submitted 6 August, 2023; originally announced August 2023.

Super-resolving sparse observations in partial differential equations: A physics-constrained convolutional neural network approach

Authors: Daniel Kelshaw, Luca Magri

Abstract: We propose the physics-constrained convolutional neural network (PC-CNN) to infer the high-resolution solution from sparse observations of spatiotemporal and nonlinear partial differential equations. Results are shown for a chaotic and turbulent fluid motion, whose solution is high-dimensional, and has fine spatiotemporal scales. We show that, by constraining prior physical knowledge in the CNN, w… ▽ More We propose the physics-constrained convolutional neural network (PC-CNN) to infer the high-resolution solution from sparse observations of spatiotemporal and nonlinear partial differential equations. Results are shown for a chaotic and turbulent fluid motion, whose solution is high-dimensional, and has fine spatiotemporal scales. We show that, by constraining prior physical knowledge in the CNN, we can infer the unresolved physical dynamics without using the high-resolution dataset in the training. This opens opportunities for super-resolution of experimental data and low-resolution simulations. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Uncovering solutions from data corrupted by systematic errors: A physics-constrained convolutional neural network approach

Authors: Daniel Kelshaw, Luca Magri

Abstract: Information on natural phenomena and engineering systems is typically contained in data. Data can be corrupted by systematic errors in models and experiments. In this paper, we propose a tool to uncover the spatiotemporal solution of the underlying physical system by removing the systematic errors from data. The tool is the physics-constrained convolutional neural network (PC-CNN), which combines… ▽ More Information on natural phenomena and engineering systems is typically contained in data. Data can be corrupted by systematic errors in models and experiments. In this paper, we propose a tool to uncover the spatiotemporal solution of the underlying physical system by removing the systematic errors from data. The tool is the physics-constrained convolutional neural network (PC-CNN), which combines information from both the systems governing equations and data. We focus on fundamental phenomena that are modelled by partial differential equations, such as linear convection, Burgers equation, and two-dimensional turbulence. First, we formulate the problem, describe the physics-constrained convolutional neural network, and parameterise the systematic error. Second, we uncover the solutions from data corrupted by large multimodal systematic errors. Third, we perform a parametric study for different systematic errors. We show that the method is robust. Fourth, we analyse the physical properties of the uncovered solutions. We show that the solutions inferred from the PC-CNN are physical, in contrast to the data corrupted by systematic errors that does not fulfil the governing equations. This work opens opportunities for removing epistemic errors from models, and systematic errors from measurements. △ Less

Submitted 19 June, 2023; v1 submitted 7 June, 2023; originally announced June 2023.

Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filter

Authors: Andrea Nóvoa, Alberto Racca, Luca Magri

Abstract: Because of physical assumptions and numerical approximations, low-order models are affected by uncertainties in the state and parameters, and by model biases. Model biases, also known as model errors or systematic errors, are difficult to infer because they are `unknown unknowns', i.e., we do not necessarily know their functional form a priori. With biased models, data assimilation methods may be… ▽ More Because of physical assumptions and numerical approximations, low-order models are affected by uncertainties in the state and parameters, and by model biases. Model biases, also known as model errors or systematic errors, are difficult to infer because they are `unknown unknowns', i.e., we do not necessarily know their functional form a priori. With biased models, data assimilation methods may be ill-posed because either (i) they are 'bias-unaware' because the estimators are assumed unbiased, (ii) they rely on an a priori parametric model for the bias, or (iii) they can infer model biases that are not unique for the same model and data. First, we design a data assimilation framework to perform combined state, parameter, and bias estimation. Second, we propose a mathematical solution with a sequential method, i.e., the regularized bias-aware ensemble Kalman Filter (r-EnKF), which requires a model of the bias and its gradient (i.e., the Jacobian). Third, we propose an echo state network as the model bias estimator. We derive the Jacobian of the network, and design a robust training strategy with data augmentation to accurately infer the bias in different scenarios. Fourth, we apply the r-EnKF to nonlinearly coupled oscillators (with and without time-delay) affected by different forms of bias. The r-EnKF infers in real-time parameters and states, and a unique bias. The applications that we showcase are relevant to acoustics, thermoacoustics, and vibrations; however, the r-EnKF opens new opportunities for combined state, parameter and bias estimation for real-time and on-the-fly prediction in nonlinear systems. △ Less

Submitted 20 September, 2023; v1 submitted 7 June, 2023; originally announced June 2023.

Comments: 22 Figures

A machine learning approach to the prediction of heat-transfer coefficients in micro-channels

Authors: Tullio Traverso, Francesco Coletti, Luca Magri, Tassos G. Karayiannis, Omar K. Matar

Abstract: The accurate prediction of the two-phase heat transfer coefficient (HTC) as a function of working fluids, channel geometries and process conditions is key to the optimal design and operation of compact heat exchangers. Advances in artificial intelligence research have recently boosted the application of machine learning (ML) algorithms to obtain data-driven surrogate models for the HTC. For most s… ▽ More The accurate prediction of the two-phase heat transfer coefficient (HTC) as a function of working fluids, channel geometries and process conditions is key to the optimal design and operation of compact heat exchangers. Advances in artificial intelligence research have recently boosted the application of machine learning (ML) algorithms to obtain data-driven surrogate models for the HTC. For most supervised learning algorithms, the task is that of a nonlinear regression problem. Despite the fact that these models have been proven capable of outperforming traditional empirical correlations, they have key limitations such as overfitting the data, the lack of uncertainty estimation, and interpretability of the results. To address these limitations, in this paper, we use a multi-output Gaussian process regression (GPR) to estimate the HTC in microchannels as a function of the mass flow rate, heat flux, system pressure and channel diameter and length. The model is trained using the Brunel Two-Phase Flow database of high-fidelity experimental data. The advantages of GPR are data efficiency, the small number of hyperparameters to be trained (typically of the same order of the number of input dimensions), and the automatic trade-off between data fit and model complexity guaranteed by the maximization of the marginal likelihood (Bayesian approach). Our paper proposes research directions to improve the performance of the GPR-based model in extrapolation. △ Less

Submitted 28 May, 2023; originally announced May 2023.

Comments: 7 pages, 2 figures, to be published in the proceedings of the 17th International Heat Transfer Conference 2023 (IHTC-17)

Data-driven modelling for drop size distributions

Authors: Tullio Traverso, Thomas Abadie, Omar K. Matar, Luca Magri

Abstract: The prediction of the drop size distribution (DSD) resulting from liquid atomization is key to the optimization of multi-phase flows, from gas-turbine propulsion, through agriculture, to healthcare. Obtaining high-fidelity data of liquid atomization, either experimentally or numerically, is expensive, which makes the exploration of the design space difficult. First, to tackle these challenges, we… ▽ More The prediction of the drop size distribution (DSD) resulting from liquid atomization is key to the optimization of multi-phase flows, from gas-turbine propulsion, through agriculture, to healthcare. Obtaining high-fidelity data of liquid atomization, either experimentally or numerically, is expensive, which makes the exploration of the design space difficult. First, to tackle these challenges, we propose a framework to predict the DSD of a liquid spray based on data as a function of the spray angle, the Reynolds number, and the Weber number. Second, to guide the design of liquid atomizers, the model accurately predicts the volume of fluid contained in drops of specific sizes whilst providing uncertainty estimation. To do so, we propose a Gaussian process regression (GPR) model, which infers the DSD and its uncertainty form the knowledge of its integrals, and of its first moment, i.e., the mean drop diameter. Third, we deploy multiple GPR models to estimate these quantities at arbitrary points of the design space from data obtained from a large number of numerical simulations of a flat fan spray. The kernel used for reconstructing the DSD incorporates prior physical knowledge, which enables the prediction of sharply peaked and heavy-tailed distributions. Fourth, we compare our method with a benchmark approach, which estimates the DSD by interpolating the frequency polygon of the binned drops with a GPR. We show that our integral approach is significantly more accurate, especially in the tail of the distribution (i.e., large, rare drops), and it reduces the bias of the density estimator by up to ten times. Finally, we discuss physical aspects of the model's predictions and interpret them against experimental results from the literature. This work opens opportunities for modelling drop size distribution in multiphase flows from data. △ Less

Submitted 29 May, 2023; originally announced May 2023.

Comments: 18 pages, 9 figures

Hard-constrained neural networks for modelling nonlinear acoustics

Authors: Defne Ege Ozan, Luca Magri

Abstract: We model acoustic dynamics in space and time from synthetic sensor data. The tasks are (i) to predict and extrapolate the spatiotemporal dynamics, and (ii) reconstruct the acoustic state from partial observations. To achieve this, we develop acoustic neural networks that learn from sensor data, whilst being constrained by prior knowledge on acoustic and wave physics by both informing the training… ▽ More We model acoustic dynamics in space and time from synthetic sensor data. The tasks are (i) to predict and extrapolate the spatiotemporal dynamics, and (ii) reconstruct the acoustic state from partial observations. To achieve this, we develop acoustic neural networks that learn from sensor data, whilst being constrained by prior knowledge on acoustic and wave physics by both informing the training and constraining parts of the network's architecture as an inductive bias. First, we show that standard feedforward neural networks are unable to extrapolate in time, even in the simplest case of periodic oscillations. Second, we constrain the prior knowledge on acoustics in increasingly effective ways by (i) employing periodic activations (periodically activated neural networks); (ii) informing the training of the networks with a penalty term that favours solutions that fulfil the governing equations (soft-constrained); (iii) constraining the architecture in a physically-motivated solution space (hard-constrained); and (iv) combination of these. Third, we apply the networks on two testcases for two tasks in nonlinear regimes, from periodic to chaotic oscillations. The first testcase is a twin experiment, in which the data is produced by a prototypical time-delayed model. In the second testcase, the data is generated by a higher-fidelity model with mean-flow effects and a kinematic model for the flame source. We find that (i) constraining the physics in the architecture improves interpolation whilst requiring smaller network sizes, (ii) extrapolation in time is achieved by periodic activations, and (iii) velocity can be reconstructed accurately from only pressure measurements with a combination of physics-based hard and soft constraints. In and beyond acoustics, this work opens strategies for constraining the physics in the architecture, rather than the training. △ Less

Submitted 18 September, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Short and Straight: Geodesics on Differentiable Manifolds

Authors: Daniel Kelshaw, Luca Magri

Abstract: Manifolds discovered by machine learning models provide a compact representation of the underlying data. Geodesics on these manifolds define locally length-minimising curves and provide a notion of distance, which are key for reduced-order modelling, statistical inference, and interpolation. In this work, we first analyse existing methods for computing length-minimising geodesics. We find that the… ▽ More Manifolds discovered by machine learning models provide a compact representation of the underlying data. Geodesics on these manifolds define locally length-minimising curves and provide a notion of distance, which are key for reduced-order modelling, statistical inference, and interpolation. In this work, we first analyse existing methods for computing length-minimising geodesics. We find that these are not suitable for obtaining valid paths, and thus, geodesic distances. We remedy these shortcomings by leveraging numerical tools from differential geometry, which provide the means to obtain Hamiltonian-conserving geodesics. Second, we propose a model-based parameterisation for distance fields and geodesic flows on continuous manifolds. Our approach exploits a manifold-aware extension to the Eikonal equation, eliminating the need for approximations or discretisation. Finally, we develop a curvature-based training mechanism, sampling and scaling points in regions of the manifold exhibiting larger values of the Ricci scalar. This sampling and scaling approach ensures that we capture regions of the manifold subject to higher degrees of geodesic deviation. Our proposed methods provide principled means to compute valid geodesics and geodesic distances on manifolds. This work opens opportunities for latent-space interpolation, optimal control, and distance computation on differentiable manifolds. △ Less

Submitted 24 May, 2023; originally announced May 2023.

Reconstruction, forecasting, and stability of chaotic dynamics from partial data

Authors: Elise Özalp, Georgios Margazoglou, Luca Magri

Abstract: The forecasting and computation of the stability of chaotic systems from partial observations are tasks for which traditional equation-based methods may not be suitable. In this computational paper, we propose data-driven methods to (i) infer the dynamics of unobserved (hidden) chaotic variables (full-state reconstruction); (ii) time forecast the evolution of the full state; and (iii) infer the st… ▽ More The forecasting and computation of the stability of chaotic systems from partial observations are tasks for which traditional equation-based methods may not be suitable. In this computational paper, we propose data-driven methods to (i) infer the dynamics of unobserved (hidden) chaotic variables (full-state reconstruction); (ii) time forecast the evolution of the full state; and (iii) infer the stability properties of the full state. The tasks are performed with long short-term memory (LSTM) networks, which are trained with observations (data) limited to only part of the state: (i) the low-to-high resolution LSTM (LH-LSTM), which takes partial observations as training input, and requires access to the full system state when computing the loss; and (ii) the physics-informed LSTM (PI-LSTM), which is designed to combine partial observations with the integral formulation of the dynamical system's evolution equations. First, we derive the Jacobian of the LSTMs. Second, we analyse a chaotic partial differential equation, the Kuramoto-Sivashinsky (KS), and the Lorenz-96 system. We show that the proposed networks can forecast the hidden variables, both time-accurately and statistically. The Lyapunov exponents and covariant Lyapunov vectors, which characterize the stability of the chaotic attractors, are correctly inferred from partial observations. Third, the PI-LSTM outperforms the LH-LSTM by successfully reconstructing the hidden chaotic dynamics when the input dimension is smaller or similar to the Kaplan-Yorke dimension of the attractor. This work opens new opportunities for reconstructing the full state, inferring hidden variables, and computing the stability of chaotic systems from partial data. △ Less

Submitted 18 August, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Journal ref: Chaos 33, 093107 (2023)

Quantum Multi-Model Fitting

Authors: Matteo Farina, Luca Magri, Willi Menapace, Elisa Ricci, Vladislav Golyanik, Federica Arrigoni

Abstract: Geometric model fitting is a challenging but fundamental computer vision problem. Recently, quantum optimization has been shown to enhance robust fitting for the case of a single model, while leaving the question of multi-model fitting open. In response to this challenge, this paper shows that the latter case can significantly benefit from quantum hardware and proposes the first quantum approach t… ▽ More Geometric model fitting is a challenging but fundamental computer vision problem. Recently, quantum optimization has been shown to enhance robust fitting for the case of a single model, while leaving the question of multi-model fitting open. In response to this challenge, this paper shows that the latter case can significantly benefit from quantum hardware and proposes the first quantum approach to multi-model fitting (MMF). We formulate MMF as a problem that can be efficiently sampled by modern adiabatic quantum computers without the relaxation of the objective function. We also propose an iterative and decomposed version of our method, which supports real-world-sized problems. The experimental evaluation demonstrates promising results on a variety of datasets. The source code is available at: https://github.com/FarinaMatteo/qmmf. △ Less

Submitted 27 March, 2023; originally announced March 2023.

Comments: In Computer Vision and Pattern Recognition (CVPR) 2023; Highlight

Data-driven stability analysis of a chaotic time-delayed system

Authors: Georgios Margazoglou, Luca Magri

Abstract: Systems with time-delayed chaotic dynamics are common in nature, from control theory to aeronautical propulsion. The overarching objective of this paper is to compute the stability properties of a chaotic dynamical system, which is time-delayed. The stability analysis is based only on data. We employ the echo state network (ESN), a type of recurrent neural network, and train it on timeseries of a… ▽ More Systems with time-delayed chaotic dynamics are common in nature, from control theory to aeronautical propulsion. The overarching objective of this paper is to compute the stability properties of a chaotic dynamical system, which is time-delayed. The stability analysis is based only on data. We employ the echo state network (ESN), a type of recurrent neural network, and train it on timeseries of a prototypical time-delayed nonlinear thermoacoustic system. By running the trained ESN autonomously, we show that it can reproduce (i) the long-term statistics of the thermoacoustic system's variables, (ii) the physical portion of the Lyapunov spectrum, and (iii) the statistics of the finite-time Lyapunov exponents. This work opens up the possibility to infer stability properties of time-delayed systems from experimental observations. △ Less

Submitted 14 February, 2023; originally announced March 2023.

Physics-Informed Long Short-Term Memory for Forecasting and Reconstruction of Chaos

Authors: Elise Özalp, Georgios Margazoglou, Luca Magri

Abstract: We present the Physics-Informed Long Short-Term Memory (PI-LSTM) network to reconstruct and predict the evolution of unmeasured variables in a chaotic system. The training is constrained by a regularization term, which penalizes solutions that violate the system's governing equations. The network is showcased on the Lorenz-96 model, a prototypical chaotic dynamical system, for a varying number of… ▽ More We present the Physics-Informed Long Short-Term Memory (PI-LSTM) network to reconstruct and predict the evolution of unmeasured variables in a chaotic system. The training is constrained by a regularization term, which penalizes solutions that violate the system's governing equations. The network is showcased on the Lorenz-96 model, a prototypical chaotic dynamical system, for a varying number of variables to reconstruct. First, we show the PI-LSTM architecture and explain how to constrain the differential equations, which is a non-trivial task in LSTMs. Second, the PI-LSTM is numerically evaluated in the long-term autonomous evolution to study its ergodic properties. We show that it correctly predicts the statistics of the unmeasured variables, which cannot be achieved without the physical constraint. Third, we compute the Lyapunov exponents of the network to infer the key stability properties of the chaotic system. For reconstruction purposes, adding the physics-informed loss qualitatively enhances the dynamical behaviour of the network, compared to a data-driven only training. This is quantified by the agreement of the Lyapunov exponents. This work opens up new opportunities for state reconstruction and learning of the dynamics of nonlinear systems. △ Less

Submitted 3 February, 2023; originally announced February 2023.

Bayesian optimization of the layout of wind farms with a high-fidelity surrogate model

Authors: Nikolaos Bempedelis, Luca Magri

Abstract: We introduce a gradient-free data-driven framework for optimizing the power output of a wind farm based on a Bayesian approach and large-eddy simulations. In contrast with conventional wind farm layout optimization strategies, which make use of simple wake models, the proposed framework accounts for complex flow phenomena such as wake meandering, local speed-ups and the interaction of the wind tur… ▽ More We introduce a gradient-free data-driven framework for optimizing the power output of a wind farm based on a Bayesian approach and large-eddy simulations. In contrast with conventional wind farm layout optimization strategies, which make use of simple wake models, the proposed framework accounts for complex flow phenomena such as wake meandering, local speed-ups and the interaction of the wind turbines with the atmospheric flow. The capabilities of the framework are demonstrated for the case of a small wind farm consisting of five wind turbines. It is shown that it can find optimal designs within a few iterations, while leveraging the above phenomena to deliver increased wind farm performance. △ Less

Submitted 2 February, 2023; originally announced February 2023.

Convolutional autoencoder for the spatiotemporal latent representation of turbulence

Authors: Nguyen Anh Khoa Doan, Alberto Racca, Luca Magri

Abstract: Turbulence is characterised by chaotic dynamics and a high-dimensional state space, which make this phenomenon challenging to predict. However, turbulent flows are often characterised by coherent spatiotemporal structures, such as vortices or large-scale modes, which can help obtain a latent description of turbulent flows. However, current approaches are often limited by either the need to use som… ▽ More Turbulence is characterised by chaotic dynamics and a high-dimensional state space, which make this phenomenon challenging to predict. However, turbulent flows are often characterised by coherent spatiotemporal structures, such as vortices or large-scale modes, which can help obtain a latent description of turbulent flows. However, current approaches are often limited by either the need to use some form of thresholding on quantities defining the isosurfaces to which the flow structures are associated or the linearity of traditional modal flow decomposition approaches, such as those based on proper orthogonal decomposition. This problem is exacerbated in flows that exhibit extreme events, which are rare and sudden changes in a turbulent state. The goal of this paper is to obtain an efficient and accurate reduced-order latent representation of a turbulent flow that exhibits extreme events. Specifically, we employ a three-dimensional multiscale convolutional autoencoder (CAE) to obtain such latent representation. We apply it to a three-dimensional turbulent flow. We show that the Multiscale CAE is efficient, requiring less than 10% degrees of freedom than proper orthogonal decomposition for compressing the data and is able to accurately reconstruct flow states related to extreme events. The proposed deep learning architecture opens opportunities for nonlinear reduced-order modeling of turbulent flows from data. △ Less

Submitted 20 June, 2023; v1 submitted 31 January, 2023; originally announced January 2023.

Indirect noise from weakly reacting inhomogeneities

Authors: Animesh Jain, Andrea Giusti, Luca Magri

Abstract: Indirect noise is a significant contributor to aircraft engine noise, which needs to be minimized in the design of aircraft engines. Indirect noise is caused by the acceleration of flow inhomogeneities through a nozzle. High-fidelity simulations showed that some flow inhomogeneities can be chemically reacting when they leave the combustor and enter the nozzle (Giusti et al., 2019). The state-of-ar… ▽ More Indirect noise is a significant contributor to aircraft engine noise, which needs to be minimized in the design of aircraft engines. Indirect noise is caused by the acceleration of flow inhomogeneities through a nozzle. High-fidelity simulations showed that some flow inhomogeneities can be chemically reacting when they leave the combustor and enter the nozzle (Giusti et al., 2019). The state-of-art models, however, are limited to chemically non-reacting (frozen) flows. In this work, first, we propose a low-order model to predict indirect noise in nozzle flows with reacting inhomogeneities. Second, we identify the physical sources of sound, which generate indirect noise via two physical mechanisms: (i) chemical reaction generates compositional perturbations, thereby adding to compositional noise; and (ii) exothermic reaction generates entropy perturbations. Third, we numerically compute the nozzle transfer functions for different frequency ranges (Helmholtz numbers) and reaction rates (Damköhler numbers) in subsonic flows with hydrogen and methane inhomogeneities. Fourth, we extend the model to supersonic flows. We find that hydrogen inhomogeneities have a larger impact to indirect noise than methane inhomogeneities. Both the Damköhler number and the Helmholtz number markedly influence the phase and magnitude of the transmitted and reflected waves, which affect the sound generation and thermoacoustic stability. This work provides a physics-based low-order model, which can open new opportunities for predicting noise emissions and instabilities in aeronautical gas turbines with multi-physics flows. △ Less

Submitted 30 January, 2023; originally announced January 2023.

Modelling spatiotemporal turbulent dynamics with the convolutional autoencoder echo state network

Authors: Alberto Racca, Nguyen Anh Khoa Doan, Luca Magri

Abstract: The spatiotemporal dynamics of turbulent flows is chaotic and difficult to predict. This makes the design of accurate and stable reduced-order models challenging. The overarching objective of this paper is to propose a nonlinear decomposition of the turbulent state for a reduced-order representation of the dynamics. We divide the turbulent flow into a spatial problem and a temporal problem. First,… ▽ More The spatiotemporal dynamics of turbulent flows is chaotic and difficult to predict. This makes the design of accurate and stable reduced-order models challenging. The overarching objective of this paper is to propose a nonlinear decomposition of the turbulent state for a reduced-order representation of the dynamics. We divide the turbulent flow into a spatial problem and a temporal problem. First, we compute the latent space, which is the manifold onto which the turbulent dynamics live (i.e., it is a numerical approximation of the turbulent attractor). The latent space is found by a series of nonlinear filtering operations, which are performed by a convolutional autoencoder (CAE). The CAE provides the decomposition in space. Second, we predict the time evolution of the turbulent state in the latent space, which is performed by an echo state network (ESN). The ESN provides the decomposition in time. Third, by assembling the CAE and the ESN, we obtain an autonomous dynamical system: the convolutional autoncoder echo state network (CAE-ESN). This is the reduced-order model of the turbulent flow. We test the CAE-ESN on a two-dimensional flow. We show that, after training, the CAE-ESN (i) finds a latent-space representation of the turbulent flow that has less than 1% of the degrees of freedom than the physical space; (ii) time-accurately and statistically predicts the flow in both quasiperiodic and turbulent regimes; (iii) is robust for different flow regimes (Reynolds numbers); and (iv) takes less than 1% of computational time to predict the turbulent flow than solving the governing equations. This work opens up new possibilities for nonlinear decompositions and reduced-order modelling of turbulent flows from data. △ Less

Submitted 22 November, 2022; v1 submitted 21 November, 2022; originally announced November 2022.

On interpretability and proper latent decomposition of autoencoders

Authors: Luca Magri, Anh Khoa Doan

Abstract: The dynamics of a turbulent flow tend to occupy only a portion of the phase space at a statistically stationary regime. From a dynamical systems point of view, this portion is the attractor. The knowledge of the turbulent attractor is useful for two purposes, at least: (i) We can gain physical insight into turbulence (what is the shape and geometry of the attractor?), and (ii) it provides the mini… ▽ More The dynamics of a turbulent flow tend to occupy only a portion of the phase space at a statistically stationary regime. From a dynamical systems point of view, this portion is the attractor. The knowledge of the turbulent attractor is useful for two purposes, at least: (i) We can gain physical insight into turbulence (what is the shape and geometry of the attractor?), and (ii) it provides the minimal number of degrees of freedom to accurately describe the turbulent dynamics. Autoencoders enable the computation of an optimal latent space, which is a low-order representation of the dynamics. If properly trained and correctly designed, autoencoders can learn an approximation of the turbulent attractor, as shown by Doan, Racca and Magri (2022). In this paper, we theoretically interpret the transformations of an autoencoder. First, we remark that the latent space is a curved manifold with curvilinear coordinates, which can be analyzed with simple tools from Riemann geometry. Second, we characterize the geometrical properties of the latent space. We mathematically derive the metric tensor, which provides a mathematical description of the manifold. Third, we propose a method -- proper latent decomposition (PLD) -- that generalizes proper orthogonal decomposition of turbulent flows on the autoencoder latent space. This decomposition finds the dominant directions in the curved latent space. This theoretical work opens up computational opportunities for interpreting autoencoders and creating reduced-order models of turbulent flows. △ Less

Submitted 2 December, 2022; v1 submitted 15 November, 2022; originally announced November 2022.

Comments: 9 pages, 2 figures, Proceedings of the Summer Program, Center for Turbulence Research, Stanford University

Physics-Informed CNNs for Super-Resolution of Sparse Observations on Dynamical Systems

Authors: Daniel Kelshaw, Georgios Rigas, Luca Magri

Abstract: In the absence of high-resolution samples, super-resolution of sparse observations on dynamical systems is a challenging problem with wide-reaching applications in experimental settings. We showcase the application of physics-informed convolutional neural networks for super-resolution of sparse observations on grids. Results are shown for the chaotic-turbulent Kolmogorov flow, demonstrating the po… ▽ More In the absence of high-resolution samples, super-resolution of sparse observations on dynamical systems is a challenging problem with wide-reaching applications in experimental settings. We showcase the application of physics-informed convolutional neural networks for super-resolution of sparse observations on grids. Results are shown for the chaotic-turbulent Kolmogorov flow, demonstrating the potential of this method for resolving finer scales of turbulence when compared with classic interpolation methods, and thus effectively reconstructing missing physics. △ Less

Submitted 7 November, 2022; v1 submitted 31 October, 2022; originally announced October 2022.

Comments: Published in NeurIPS 2022: Machine Learning and the Physical Sciences Workshop. Code at https://github.com/magrilab/pisr. arXiv admin note: text overlap with arXiv:2210.16215

Physics-Informed Convolutional Neural Networks for Corruption Removal on Dynamical Systems

Authors: Daniel Kelshaw, Luca Magri

Abstract: Measurements on dynamical systems, experimental or otherwise, are often subjected to inaccuracies capable of introducing corruption; removal of which is a problem of fundamental importance in the physical sciences. In this work we propose physics-informed convolutional neural networks for stationary corruption removal, providing the means to extract physical solutions from data, given access to pa… ▽ More Measurements on dynamical systems, experimental or otherwise, are often subjected to inaccuracies capable of introducing corruption; removal of which is a problem of fundamental importance in the physical sciences. In this work we propose physics-informed convolutional neural networks for stationary corruption removal, providing the means to extract physical solutions from data, given access to partial ground-truth observations at collocation points. We showcase the methodology for 2D incompressible Navier-Stokes equations in the chaotic-turbulent flow regime, demonstrating robustness to modality and magnitude of corruption. △ Less

Submitted 7 November, 2022; v1 submitted 28 October, 2022; originally announced October 2022.

Comments: Published in NeurIPS 2022: Machine Learning and the Physical Sciences Workshop. Code at http://github.com/magriLab/PICR

Stability analysis of chaotic systems from data

Authors: Georgios Margazoglou, Luca Magri

Abstract: The prediction of the temporal dynamics of chaotic systems is challenging because infinitesimal perturbations grow exponentially. The analysis of the dynamics of infinitesimal perturbations is the subject of stability analysis. In stability analysis, we linearize the equations of the dynamical system around a reference point, and compute the properties of the tangent space (i.e., the Jacobian). Th… ▽ More The prediction of the temporal dynamics of chaotic systems is challenging because infinitesimal perturbations grow exponentially. The analysis of the dynamics of infinitesimal perturbations is the subject of stability analysis. In stability analysis, we linearize the equations of the dynamical system around a reference point, and compute the properties of the tangent space (i.e., the Jacobian). The main goal of this paper is to propose a method that learns the Jacobian, thus, the stability properties, from observables (data). First, we propose the Echo State Network (ESN) with the Recycle Validation as a tool to accurately infer the chaotic dynamics from data. Second, we mathematically derive the Jacobian of the Echo State Network, which provides the evolution of infinitesimal perturbations. Third, we analyse the stability properties of the Jacobian inferred from the ESN, and compare them with the benchmark results obtained by linearizing the equations. The ESN correctly infers the nonlinear solution, and its tangent space with negligible numerical errors. In detail, (i) the long-term statistics of the chaotic state; (ii) the covariant Lyapunov vectors; (ii) the Lyapunov spectrum; (iii) the finite-time Lyapunov exponents; (iv) and the angles between the stable, neutral, and unstable splittings of the tangent space (the degree of hyperbolicity of the attractor). This work opens up new opportunities for the computation of stability properties of nonlinear systems from data, instead of equations. △ Less

Submitted 30 March, 2023; v1 submitted 12 October, 2022; originally announced October 2022.

A physics-based domain adaptation framework for modelling and forecasting building energy systems

Authors: Zack Xuereb Conti, Ruchi Choudhary, Luca Magri

Abstract: State-of-the-art machine-learning-based models are a popular choice for modeling and forecasting energy behavior in buildings because given enough data, they are good at finding spatiotemporal patterns and structures even in scenarios where the complexity prohibits analytical descriptions. However, their architecture typically does not hold physical correspondence to mechanistic structures linked… ▽ More State-of-the-art machine-learning-based models are a popular choice for modeling and forecasting energy behavior in buildings because given enough data, they are good at finding spatiotemporal patterns and structures even in scenarios where the complexity prohibits analytical descriptions. However, their architecture typically does not hold physical correspondence to mechanistic structures linked with governing physical phenomena. As a result, their ability to successfully generalize for unobserved timesteps depends on the representativeness of the dynamics underlying the observed system in the data, which is difficult to guarantee in real-world engineering problems such as control and energy management in digital twins. In response, we present a framework that combines lumped-parameter models in the form of linear time-invariant (LTI) state-space models (SSMs) with unsupervised reduced-order modeling in a subspace-based domain adaptation (SDA) framework. SDA is a type of transfer-learning (TL) technique, typically adopted for exploiting labeled data from one domain to predict in a different but related target domain for which labeled data is limited. We introduce a novel SDA approach where instead of labeled data, we leverage the geometric structure of the LTI SSM governed by well-known heat transfer ordinary differential equations to forecast for unobserved timesteps beyond observed measurement data. Fundamentally, our approach geometrically aligns the physics-derived and data-derived embedded subspaces closer together. In this initial exploration, we evaluate the physics-based SDA framework on a demonstrative heat conduction scenario by varying the thermophysical properties of the source and target systems to demonstrate the transferability of mechanistic models from a physics-based domain to a data domain. △ Less

Submitted 2 May, 2023; v1 submitted 19 August, 2022; originally announced August 2022.

Compositional noise in nozzles with dissipation

Authors: Animesh Jain, Luca Magri

Abstract: We propose a physical model to predict indirect noise generated by the acceleration of compositional inhomogeneities in nozzles with viscous dissipation (non-isentropic nozzles). First, we derive the quasi-one-dimensional equations from the conservation laws of multicomponent flows. Second, we validate the proposed model with the experimental data available in the literature for binary mixtures of… ▽ More We propose a physical model to predict indirect noise generated by the acceleration of compositional inhomogeneities in nozzles with viscous dissipation (non-isentropic nozzles). First, we derive the quasi-one-dimensional equations from the conservation laws of multicomponent flows. Second, we validate the proposed model with the experimental data available in the literature for binary mixtures of four gases. Third, we calculate the nozzle transfer functions for different Helmholtz numbers and friction factors, in both subsonic and supersonic flows with/without shock waves. We show that friction has a significant effect on the generation of indirect noise, for which the physical mechanism is identified and explained. Fourth, we find a semi-analytical solution with path integrals, which provide an asymptotic expansion with respect to the Helmholtz number. Fifth, we introduce the compositional noise scaling factor, which is applied to quickly estimate compositional noise from the knowledge of only one single-component gas transfer function. The approximation error is less than $1\%$. The proposed low-order model provides accurate estimates of the transfer functions and physical insight into indirect noise for multicomponent gases. This opens up new possibilities to accurately predict, and understand, sound generation in gas turbines. △ Less

Submitted 4 May, 2023; v1 submitted 14 June, 2022; originally announced June 2022.

Data-driven prediction and control of extreme events in a chaotic flow

Authors: Alberto Racca, Luca Magri

Abstract: An extreme event is a sudden and violent change in the state of a nonlinear system. In fluid dynamics, extreme events can have adverse effects on the system's optimal design and operability, which calls for accurate methods for their prediction and control. In this paper, we propose a data-driven methodology for the prediction and control of extreme events in a chaotic shear flow. The approach is… ▽ More An extreme event is a sudden and violent change in the state of a nonlinear system. In fluid dynamics, extreme events can have adverse effects on the system's optimal design and operability, which calls for accurate methods for their prediction and control. In this paper, we propose a data-driven methodology for the prediction and control of extreme events in a chaotic shear flow. The approach is based on echo state networks, which are a type of reservoir computing that learn temporal correlations within a time-dependent dataset. The objective is five-fold. First, we exploit ad-hoc metrics from binary classification to analyse (i) how many of the extreme events predicted by the network actually occur in the test set (precision), and (ii) how many extreme events are missed by the network (recall). We apply a principled strategy for optimal hyperparameter selection, which is key to the networks' performance. Second, we focus on the time-accurate prediction of extreme events. We show that echo state networks are able to predict extreme events well beyond the predictability time, i.e., up to more than five Lyapunov times. Third, we focus on the long-term prediction of extreme events from a statistical point of view. By training the networks with datasets that contain non-converged statistics, we show that the networks are able to learn and extrapolate the flow's long-term statistics. In other words, the networks are able to extrapolate in time from relatively short time series. Fourth, we design a simple and effective control strategy to prevent extreme events from occurring. The control strategy decreases the occurrence of extreme events up to one order of magnitude with respect to the uncontrolled system. Finally, we analyse the robustness of the results for a range of Reynolds numbers. We show that the networks perform well across a wide range of regimes. △ Less

Submitted 25 April, 2022; originally announced April 2022.

Statistical prediction of extreme events from small datasets

Authors: Alberto Racca, Luca Magri

Abstract: We propose Echo State Networks (ESNs) to predict the statistics of extreme events in a turbulent flow. We train the ESNs on small datasets that lack information about the extreme events. We asses whether the networks are able to extrapolate from the small imperfect datasets and predict the heavy-tail statistics that describe the events. We find that the networks correctly predict the events and im… ▽ More We propose Echo State Networks (ESNs) to predict the statistics of extreme events in a turbulent flow. We train the ESNs on small datasets that lack information about the extreme events. We asses whether the networks are able to extrapolate from the small imperfect datasets and predict the heavy-tail statistics that describe the events. We find that the networks correctly predict the events and improve the statistics of the system with respect to the training data in almost all cases analysed. This opens up new possibilities for the statistical prediction of extreme events in turbulence. △ Less

Submitted 12 April, 2022; v1 submitted 20 January, 2022; originally announced January 2022.

A physical model for indirect noise in non-isentropic nozzles: Transfer functions and stability

Authors: Animesh Jain, Luca Magri

Abstract: We propose a mathematical model from physical principles to predict the sound generated in nozzles with dissipation. The focus is on the sound generated from the acceleration of temperature inhomogeneities (also known as entropy waves), which is referred to as indirect noise. First, we model the dissipation caused by flow recirculation and wall friction with a friction factor, which enables us to… ▽ More We propose a mathematical model from physical principles to predict the sound generated in nozzles with dissipation. The focus is on the sound generated from the acceleration of temperature inhomogeneities (also known as entropy waves), which is referred to as indirect noise. First, we model the dissipation caused by flow recirculation and wall friction with a friction factor, which enables us to derive quasi-one-dimensional equations from conservation laws. The model is valid for both compact nozzles and nozzles with a spatial extent. Second, the predictions from the proposed model are compared against the experimental data available in the literature. Third, we compute the nozzle transfer functions for a range of Helmholtz numbers and friction factors. It is found that the friction and the Helmholtz number have a significant effect on the gain/phase of the reflected and transmitted waves. The analysis is performed from subsonic to supersonic regimes (with and without shock waves). The acoustic transfer functions vary significantly because of non-isentropic effects and the Helmholtz number, in particular, in the subsonic-choked regime. Finally, we calculate the effect that the friction of a nozzle guide vane has on thermoacoustic stability. It is found that the friction and the Helmholtz number can change thermoacoustic stability from a linearly stable regime to a linearly unstable regime. The study opens up new possibilities for the accurate prediction of indirect noise in realistic nozzles with implications on both noise emissions and thermoacoustic stability. nozzles with implications on both noise emissions and thermoacoustic stability. △ Less

Submitted 7 February, 2022; v1 submitted 19 June, 2021; originally announced June 2021.

Comments: 27 pages, 11 figures

Gradient-free optimization of chaotic acoustics with reservoir computing

Authors: Francisco Huhn, Luca Magri

Abstract: We develop a versatile optimization method, which finds the design parameters that minimize time-averaged acoustic cost functionals. The method is gradient-free, model-informed, and data-driven with reservoir computing based on echo state networks. First, we analyse the predictive capabilities of echo state networks both in the short- and long-time prediction of the dynamics. We find that both ful… ▽ More We develop a versatile optimization method, which finds the design parameters that minimize time-averaged acoustic cost functionals. The method is gradient-free, model-informed, and data-driven with reservoir computing based on echo state networks. First, we analyse the predictive capabilities of echo state networks both in the short- and long-time prediction of the dynamics. We find that both fully data-driven and model-informed architectures learn the chaotic acoustic dynamics, both time-accurately and statistically. Informing the training with a physical reduced-order model with one acoustic mode markedly improves the accuracy and robustness of the echo state networks, whilst kee** the computational cost low. Echo state networks offer accurate predictions of the long-time dynamics, which would be otherwise expensive by integrating the governing equations to evaluate the time-averaged quantity to optimize. Second, we couple echo state networks with a Bayesian technique to explore the design thermoacoustic parameter space. The computational method is minimally intrusive. Third, we find the set of flame parameters that minimize the time-averaged acoustic energy of chaotic oscillations, which are caused by the positive feedback with a heat source, such as a flame in gas turbines or rocket motors. These oscillations are known as thermoacoustic oscillations. The optimal set of flame parameters is found with the same accuracy as brute-force grid search, but with a convergence rate that is more than one order of magnitude faster. This work opens up new possibilities for non-intrusive ("hands-off") optimization of chaotic systems, in which the cost of generating data, for example from high-fidelity simulations and experiments, is high. △ Less

Submitted 2 February, 2022; v1 submitted 17 June, 2021; originally announced June 2021.

Comments: 16 figures, 26 pages

Journal ref: Physical Review Fluids 7, 014402, 2022

Real-time thermoacoustic data assimilation

Authors: Andrea Nóvoa, Luca Magri

Abstract: Low-order thermoacoustic models are qualitatively correct, but they are typically quantitatively inaccurate. We propose a time-domain bias-aware method to make qualitatively low--order models quantitatively (more) accurate. First, we develop a Bayesian ensemble data assimilation method for a low-order model to self-adapt and self-correct any time that reference data becomes available. Second, we a… ▽ More Low-order thermoacoustic models are qualitatively correct, but they are typically quantitatively inaccurate. We propose a time-domain bias-aware method to make qualitatively low--order models quantitatively (more) accurate. First, we develop a Bayesian ensemble data assimilation method for a low-order model to self-adapt and self-correct any time that reference data becomes available. Second, we apply the methodology to infer the thermoacoustic states and heat release parameters on the fly without storing data (real-time). We perform twin experiments using synthetic acoustic pressure measurements to analyse the performance of data assimilation in all nonlinear thermoacoustic regimes, from limit cycles to chaos, and interpret the results physically. Third, we propose practical rules for thermoacoustic data assimilation. An increase, reject, inflate strategy is proposed to deal with the rich nonlinear behaviour; and physical time scales for assimilation are proposed in non-chaotic regimes (with the Nyquist-Shannon criterion) and in chaotic regimes (with the Lyapunov time). Fourth, we perform data assimilation using data from a higher-fidelity model. We introduce an echo state network to estimate in real-time the forecast bias, which is the model error of the low-fidelity model. We show that (i) the correct acoustic pressure, parameters, and model bias can be accurately inferred; (ii) the learning is robust as it can tackle large uncertainties in the observations (up to 50% the mean values); (iii) the uncertainty of the prediction and parameters is naturally part of the output; and (iv) both the time-accurate solution and statistics can be successfully inferred. Data assimilation opens up new possibility for real-time prediction of thermoacoustics by synergistically combining physical knowledge and experimental data. △ Less

Submitted 19 March, 2022; v1 submitted 11 June, 2021; originally announced June 2021.

Comments: 49 pages, 26 figures

Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics

Authors: Alberto Racca, Luca Magri

Abstract: An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network's performance. The aim of t… ▽ More An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network's performance. The aim of this paper is to assess and improve the robustness of Echo State Networks for the time-accurate prediction of chaotic solutions. The goal is three-fold. First, we investigate the robustness of routinely used validation strategies. Second, we propose the Recycle Validation, and the chaotic versions of existing validation strategies, to specifically tackle the forecasting of chaotic systems. Third, we compare Bayesian optimization with the traditional Grid Search for optimal hyperparameter selection. Numerical tests are performed on two prototypical nonlinear systems that have both chaotic and quasiperiodic solutions. Both model-free and model-informed Echo State Networks are analysed. By comparing the network's robustness in learning chaotic versus quasiperiodic solutions, we highlight fundamental challenges in learning chaotic solutions. The proposed validation strategies, which are based on the dynamical systems properties of chaotic time series, are shown to outperform the state-of-the-art validation strategies. Because the strategies are principled-they are based on chaos theory such as the Lyapunov time-they can be applied to other Recurrent Neural Networks architectures with little modification. This work opens up new possibilities for the robust design and application of Echo State Networks, and Recurrent Neural Networks, to the time-accurate prediction of chaotic systems. △ Less

Submitted 15 March, 2021; v1 submitted 9 February, 2021; originally announced March 2021.

Comments: 29 pages, 25 figures

Short- and long-term prediction of a chaotic flow: A physics-constrained reservoir computing approach

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We propose a physics-constrained machine learning method-based on reservoir computing- to time-accurately predict extreme events and long-term velocity statistics in a model of turbulent shear flow. The method leverages the strengths of two different approaches: empirical modelling based on reservoir computing, which it learns the chaotic dynamics from data only, and physical modelling based on co… ▽ More We propose a physics-constrained machine learning method-based on reservoir computing- to time-accurately predict extreme events and long-term velocity statistics in a model of turbulent shear flow. The method leverages the strengths of two different approaches: empirical modelling based on reservoir computing, which it learns the chaotic dynamics from data only, and physical modelling based on conservation laws, which extrapolates the dynamics when training data becomes unavailable. We show that the combination of the two approaches is able to accurately reproduce the velocity statistics and to predict the occurrence and amplitude of extreme events in a model of self-sustaining process in turbulence. In this flow, the extreme events are abrupt transitions from turbulent to quasi-laminar states, which are deterministic phenomena that cannot be traditionally predicted because of chaos. Furthermore, the physics-constrained machine learning method is shown to be robust with respect to noise. This work opens up new possibilities for synergistically enhancing data-driven methods with physical knowledge for the time-accurate prediction of chaotic flows. △ Less

Submitted 13 April, 2021; v1 submitted 15 February, 2021; originally announced February 2021.

Comments: 10 pages, 10 figures. arXiv admin note: substantial text overlap with arXiv:1912.10994

Automatic-differentiated Physics-Informed Echo State Network (API-ESN)

Authors: Alberto Racca, Luca Magri

Abstract: We propose the Automatic-differentiated Physics-Informed Echo State Network (API-ESN). The network is constrained by the physical equations through the reservoir's exact time-derivative, which is computed by automatic differentiation. As compared to the original Physics-Informed Echo State Network, the accuracy of the time-derivative is increased by up to seven orders of magnitude. This increased… ▽ More We propose the Automatic-differentiated Physics-Informed Echo State Network (API-ESN). The network is constrained by the physical equations through the reservoir's exact time-derivative, which is computed by automatic differentiation. As compared to the original Physics-Informed Echo State Network, the accuracy of the time-derivative is increased by up to seven orders of magnitude. This increased accuracy is key in chaotic dynamical systems, where errors grows exponentially in time. The network is showcased in the reconstruction of unmeasured (hidden) states of a chaotic system. The API-ESN eliminates a source of error, which is present in existing physics-informed echo state networks, in the computation of the time-derivative. This opens up new possibilities for an accurate reconstruction of chaotic dynamical states. △ Less

Submitted 24 March, 2021; v1 submitted 28 December, 2020; originally announced January 2021.

Comments: 7 pages, 3 figures

Auto-Encoded Reservoir Computing for Turbulence Learning

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We present an Auto-Encoded Reservoir-Computing (AE-RC) approach to learn the dynamics of a 2D turbulent flow. The AE-RC consists of an Autoencoder, which discovers an efficient manifold representation of the flow state, and an Echo State Network, which learns the time evolution of the flow in the manifold. The AE-RC is able to both learn the time-accurate dynamics of the flow and predict its first… ▽ More We present an Auto-Encoded Reservoir-Computing (AE-RC) approach to learn the dynamics of a 2D turbulent flow. The AE-RC consists of an Autoencoder, which discovers an efficient manifold representation of the flow state, and an Echo State Network, which learns the time evolution of the flow in the manifold. The AE-RC is able to both learn the time-accurate dynamics of the flow and predict its first-order statistical moments. The AE-RC approach opens up new possibilities for the spatio-temporal prediction of turbulence with machine learning. △ Less

Submitted 24 March, 2021; v1 submitted 20 December, 2020; originally announced December 2020.

Comments: 7 pages, 5 figures

Variational optimization and data assimilation in chaotic time-delayed systems with automatic-differentiated shadowing sensitivity

Authors: Nisha Chandramoorthy, Luca Magri, Qiqi Wang

Abstract: In this computational paper, we perform sensitivity analysis of long-time (or ensemble) averages in the chaotic regime using the shadowing algorithm. We introduce automatic differentiation to eliminate the tangent/adjoint equation solvers used in the shadowing algorithm. In a gradient-based optimization, we use the computed shadowing sensitivity to minimize different long-time averaged functionals… ▽ More In this computational paper, we perform sensitivity analysis of long-time (or ensemble) averages in the chaotic regime using the shadowing algorithm. We introduce automatic differentiation to eliminate the tangent/adjoint equation solvers used in the shadowing algorithm. In a gradient-based optimization, we use the computed shadowing sensitivity to minimize different long-time averaged functionals of a chaotic time-delayed system by optimal parameter selection. In combined state and parameter estimation for data assimilation, we use the computed sensitivity to predict the optimal trajectory given information from a model and data from measurements beyond the predictability time. The algorithms are applied to a thermoacoustic model. Because the computational framework is rather general, the techniques presented in this paper may be used for sensitivity analysis of ensemble averages, parameter optimization and data assimilation of other chaotic problems, where shadowing methods are applicable. △ Less

Submitted 16 November, 2020; originally announced November 2020.

Comments: 50 pages, under review

Physics-Informed Echo State Networks

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks while ensuring that their predictions do not violate physical laws. This is achieved by introducing an additional loss function during the training, which is based on the system's governing equati… ▽ More We propose a physics-informed Echo State Network (ESN) to predict the evolution of chaotic systems. Compared to conventional ESNs, the physics-informed ESNs are trained to solve supervised learning tasks while ensuring that their predictions do not violate physical laws. This is achieved by introducing an additional loss function during the training, which is based on the system's governing equations. The additional loss function penalizes non-physical predictions without the need of any additional training data. This approach is demonstrated on a chaotic Lorenz system and a truncation of the Charney-DeVore system. Compared to the conventional ESNs, the physics-informed ESNs improve the predictability horizon by about two Lyapunov times. This approach is also shown to be robust with regard to noise. The proposed framework shows the potential of using machine learning combined with prior physical knowledge to improve the time-accurate prediction of chaotic dynamical systems. △ Less

Submitted 31 October, 2020; originally announced November 2020.

Comments: 10 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:1906.11122

Journal ref: Journal of computational science, 2020

Degenerate perturbation theory in thermoacoustics: high-order sensitivities and exceptional points

Authors: Alessandro Orchini, Luca Magri, Camilo F. Silva, Georg A. Mensah, Jonas P. Moeck

Abstract: In this study, we connect concepts that have been recently developed in thermoacoustics, specifically, (i) high-order spectral perturbation theory, (ii) symmetry induced degenerate thermoacoustic modes, (iii) intrinsic thermoacoustic modes, and (iv) exceptional points. Their connection helps gain physical insight into the behaviour of the thermoacoustic spectrum when parameters of the system are v… ▽ More In this study, we connect concepts that have been recently developed in thermoacoustics, specifically, (i) high-order spectral perturbation theory, (ii) symmetry induced degenerate thermoacoustic modes, (iii) intrinsic thermoacoustic modes, and (iv) exceptional points. Their connection helps gain physical insight into the behaviour of the thermoacoustic spectrum when parameters of the system are varied. First, we extend high-order adjoint-based perturbation theory of thermoacoustic modes to the degenerate case. We provide explicit formulae for the calculation of the eigenvalue corrections to any order. These formulae are valid for self-adjoint, non-self-adjoint or even non-normal systems; therefore, they can be applied to a large range of problems, including fluid dynamics. Second, by analysing the expansion coefficients of the eigenvalue corrections as a function of a parameter of interest, we accurately estimate the radius of convergence of the power series. Third, we connect the existence of a finite radius of convergence to the existence of singularities in parameter space. We identify these singularities as exceptional points, which correspond to defective thermoacoustic eigenvalues, with infinite sensitivity to infinitesimal changes in the parameters. At an exceptional point, two eigenvalues and their associated eigenvectors coalesce. Close to an exceptional point, strong veering of the eigenvalue trajectories is observed. As demonstrated in recent work, exceptional points naturally arise in thermoacoustic systems due to the interaction between modes of acoustic and intrinsic origin. The role of exceptional points in thermoacoustic systems sheds new light on the physics and sensitivity of thermoacoustic stability, which can be leveraged for passive control by small design modifications. △ Less

Submitted 8 July, 2020; originally announced July 2020.

Comments: 30 pages, 8 figures, accepted for publication

Journal ref: Journal of Fluid Mechanics, 2020

A data-driven kinematic model of a ducted premixed flame

Authors: Hans Yu, Matthew P. Juniper, Luca Magri

Abstract: Reduced-order models of flame dynamics can be used to predict and mitigate the emergence of thermoacoustic oscillations in the design of gas turbine and rocket engines. This process is hindered by the fact that these models, although often qualitatively correct, are not usually quantitatively accurate. As automated experiments and numerical simulations produce ever-increasing quantities of data, t… ▽ More Reduced-order models of flame dynamics can be used to predict and mitigate the emergence of thermoacoustic oscillations in the design of gas turbine and rocket engines. This process is hindered by the fact that these models, although often qualitatively correct, are not usually quantitatively accurate. As automated experiments and numerical simulations produce ever-increasing quantities of data, the question arises as to how this data can be assimilated into physics-informed reduced-order models in order to render these models quantitatively accurate. In this study, we develop and test a physics-based reduced-order model of a ducted premixed flame in which the model parameters are learned from high speed videos of the flame. The experimental data is assimilated into a level-set solver using an ensemble Kalman filter. This leads to an optimally calibrated reduced-order model with quantified uncertainties, which accurately reproduces elaborate nonlinear features such as cusp formation and pinch-off. The reduced-order model continues to match the experiments after assimilation has been switched off. Further, the parameters of the model, which are extracted automatically, are shown to match the first order behavior expected on physical grounds. This study shows how reduced-order models can be updated rapidly whenever new experimental or numerical data becomes available, without the data itself having to be stored. △ Less

Submitted 4 July, 2020; originally announced July 2020.

Comments: 16 pages, 8 figures

First-principles machine learning modelling of COVID-19

Authors: Luca Magri, Nguyen Anh Khoa Doan

Abstract: The coronavirus disease 2019 (COVID-19) has changed the world since the World Health Organization declared its outbreak on 30th January 2020, recognizing the outbreak as a pandemic on 11th March 2020. As often said by politicians and scientific advisors, the objective is "to flatten the curve", or "push the peak down", or similar wording, of the virus spreading. Central to the official advice are… ▽ More The coronavirus disease 2019 (COVID-19) has changed the world since the World Health Organization declared its outbreak on 30th January 2020, recognizing the outbreak as a pandemic on 11th March 2020. As often said by politicians and scientific advisors, the objective is "to flatten the curve", or "push the peak down", or similar wording, of the virus spreading. Central to the official advice are mathematical models and data, which provide estimates on the evolution of the number of infected, recovered and deaths. The accuracy of the models is improved day by day by inferring the contact, recovery, and death rates from data (confirmed cases). A data-driven model trained with {\it both} data {\it and} first principles is proposed. The model can quickly be re-trained any time that new data becomes available. The method can be applied to more detailed epidemic models with virtually no conceptual modification. △ Less

Submitted 20 April, 2020; originally announced April 2020.

Comments: 39 pages, 53 figures

Learning ergodic averages in chaotic systems

Authors: Francisco Huhn, Luca Magri

Abstract: We propose a physics-informed machine learning method to predict the time average of a chaotic attractor. The method is based on the hybrid echo state network (hESN). We assume that the system is ergodic, so the time average is equal to the ergodic average. Compared to conventional echo state networks (ESN) (purely data-driven), the hESN uses additional information from an incomplete, or imperfect… ▽ More We propose a physics-informed machine learning method to predict the time average of a chaotic attractor. The method is based on the hybrid echo state network (hESN). We assume that the system is ergodic, so the time average is equal to the ergodic average. Compared to conventional echo state networks (ESN) (purely data-driven), the hESN uses additional information from an incomplete, or imperfect, physical model. We evaluate the performance of the hESN and compare it to that of an ESN. This approach is demonstrated on a chaotic time-delayed thermoacoustic system, where the inclusion of a physical model significantly improves the accuracy of the prediction, reducing the relative error from 48% to 7%. This improvement is obtained at the low extra cost of solving two ordinary differential equations. This framework shows the potential of using machine learning techniques combined with prior physical knowledge to improve the prediction of time-averaged quantities in chaotic systems. △ Less

Submitted 7 April, 2020; v1 submitted 9 January, 2020; originally announced January 2020.

Comments: 9 pages, 4 figures

Learning Hidden States in a Chaotic System: A Physics-Informed Echo State Network Approach

Authors: Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri

Abstract: We extend the Physics-Informed Echo State Network (PI-ESN) framework to reconstruct the evolution of an unmeasured state (hidden state) in a chaotic system. The PI-ESN is trained by using (i) data, which contains no information on the unmeasured state, and (ii) the physical equations of a prototypical chaotic dynamical system. Non-noisy and noisy datasets are considered. First, it is shown that th… ▽ More We extend the Physics-Informed Echo State Network (PI-ESN) framework to reconstruct the evolution of an unmeasured state (hidden state) in a chaotic system. The PI-ESN is trained by using (i) data, which contains no information on the unmeasured state, and (ii) the physical equations of a prototypical chaotic dynamical system. Non-noisy and noisy datasets are considered. First, it is shown that the PI-ESN can accurately reconstruct the unmeasured state. Second, the reconstruction is shown to be robust with respect to noisy data, which means that the PI-ESN acts as a denoiser. This paper opens up new possibilities for leveraging the synergy between physical knowledge and machine learning to enhance the reconstruction and prediction of unmeasured states in chaotic dynamical systems. △ Less

Submitted 7 April, 2020; v1 submitted 6 January, 2020; originally announced January 2020.

Comments: 7 pages, 4 figures