-
Reconstruction, forecasting, and stability of chaotic dynamics from partial data
Authors:
Elise Özalp,
Georgios Margazoglou,
Luca Magri
Abstract:
The forecasting and computation of the stability of chaotic systems from partial observations are tasks for which traditional equation-based methods may not be suitable. In this computational paper, we propose data-driven methods to (i) infer the dynamics of unobserved (hidden) chaotic variables (full-state reconstruction); (ii) time forecast the evolution of the full state; and (iii) infer the st…
▽ More
The forecasting and computation of the stability of chaotic systems from partial observations are tasks for which traditional equation-based methods may not be suitable. In this computational paper, we propose data-driven methods to (i) infer the dynamics of unobserved (hidden) chaotic variables (full-state reconstruction); (ii) time forecast the evolution of the full state; and (iii) infer the stability properties of the full state. The tasks are performed with long short-term memory (LSTM) networks, which are trained with observations (data) limited to only part of the state: (i) the low-to-high resolution LSTM (LH-LSTM), which takes partial observations as training input, and requires access to the full system state when computing the loss; and (ii) the physics-informed LSTM (PI-LSTM), which is designed to combine partial observations with the integral formulation of the dynamical system's evolution equations. First, we derive the Jacobian of the LSTMs. Second, we analyse a chaotic partial differential equation, the Kuramoto-Sivashinsky (KS), and the Lorenz-96 system. We show that the proposed networks can forecast the hidden variables, both time-accurately and statistically. The Lyapunov exponents and covariant Lyapunov vectors, which characterize the stability of the chaotic attractors, are correctly inferred from partial observations. Third, the PI-LSTM outperforms the LH-LSTM by successfully reconstructing the hidden chaotic dynamics when the input dimension is smaller or similar to the Kaplan-Yorke dimension of the attractor. This work opens new opportunities for reconstructing the full state, inferring hidden variables, and computing the stability of chaotic systems from partial data.
△ Less
Submitted 18 August, 2023; v1 submitted 24 May, 2023;
originally announced May 2023.
-
Data-driven stability analysis of a chaotic time-delayed system
Authors:
Georgios Margazoglou,
Luca Magri
Abstract:
Systems with time-delayed chaotic dynamics are common in nature, from control theory to aeronautical propulsion. The overarching objective of this paper is to compute the stability properties of a chaotic dynamical system, which is time-delayed. The stability analysis is based only on data. We employ the echo state network (ESN), a type of recurrent neural network, and train it on timeseries of a…
▽ More
Systems with time-delayed chaotic dynamics are common in nature, from control theory to aeronautical propulsion. The overarching objective of this paper is to compute the stability properties of a chaotic dynamical system, which is time-delayed. The stability analysis is based only on data. We employ the echo state network (ESN), a type of recurrent neural network, and train it on timeseries of a prototypical time-delayed nonlinear thermoacoustic system. By running the trained ESN autonomously, we show that it can reproduce (i) the long-term statistics of the thermoacoustic system's variables, (ii) the physical portion of the Lyapunov spectrum, and (iii) the statistics of the finite-time Lyapunov exponents. This work opens up the possibility to infer stability properties of time-delayed systems from experimental observations.
△ Less
Submitted 14 February, 2023;
originally announced March 2023.
-
Physics-Informed Long Short-Term Memory for Forecasting and Reconstruction of Chaos
Authors:
Elise Özalp,
Georgios Margazoglou,
Luca Magri
Abstract:
We present the Physics-Informed Long Short-Term Memory (PI-LSTM) network to reconstruct and predict the evolution of unmeasured variables in a chaotic system. The training is constrained by a regularization term, which penalizes solutions that violate the system's governing equations. The network is showcased on the Lorenz-96 model, a prototypical chaotic dynamical system, for a varying number of…
▽ More
We present the Physics-Informed Long Short-Term Memory (PI-LSTM) network to reconstruct and predict the evolution of unmeasured variables in a chaotic system. The training is constrained by a regularization term, which penalizes solutions that violate the system's governing equations. The network is showcased on the Lorenz-96 model, a prototypical chaotic dynamical system, for a varying number of variables to reconstruct. First, we show the PI-LSTM architecture and explain how to constrain the differential equations, which is a non-trivial task in LSTMs. Second, the PI-LSTM is numerically evaluated in the long-term autonomous evolution to study its ergodic properties. We show that it correctly predicts the statistics of the unmeasured variables, which cannot be achieved without the physical constraint. Third, we compute the Lyapunov exponents of the network to infer the key stability properties of the chaotic system. For reconstruction purposes, adding the physics-informed loss qualitatively enhances the dynamical behaviour of the network, compared to a data-driven only training. This is quantified by the agreement of the Lyapunov exponents. This work opens up new opportunities for state reconstruction and learning of the dynamics of nonlinear systems.
△ Less
Submitted 3 February, 2023;
originally announced February 2023.
-
Stability analysis of chaotic systems from data
Authors:
Georgios Margazoglou,
Luca Magri
Abstract:
The prediction of the temporal dynamics of chaotic systems is challenging because infinitesimal perturbations grow exponentially. The analysis of the dynamics of infinitesimal perturbations is the subject of stability analysis. In stability analysis, we linearize the equations of the dynamical system around a reference point, and compute the properties of the tangent space (i.e., the Jacobian). Th…
▽ More
The prediction of the temporal dynamics of chaotic systems is challenging because infinitesimal perturbations grow exponentially. The analysis of the dynamics of infinitesimal perturbations is the subject of stability analysis. In stability analysis, we linearize the equations of the dynamical system around a reference point, and compute the properties of the tangent space (i.e., the Jacobian). The main goal of this paper is to propose a method that learns the Jacobian, thus, the stability properties, from observables (data). First, we propose the Echo State Network (ESN) with the Recycle Validation as a tool to accurately infer the chaotic dynamics from data. Second, we mathematically derive the Jacobian of the Echo State Network, which provides the evolution of infinitesimal perturbations. Third, we analyse the stability properties of the Jacobian inferred from the ESN, and compare them with the benchmark results obtained by linearizing the equations. The ESN correctly infers the nonlinear solution, and its tangent space with negligible numerical errors. In detail, (i) the long-term statistics of the chaotic state; (ii) the covariant Lyapunov vectors; (ii) the Lyapunov spectrum; (iii) the finite-time Lyapunov exponents; (iv) and the angles between the stable, neutral, and unstable splittings of the tangent space (the degree of hyperbolicity of the attractor). This work opens up new opportunities for the computation of stability properties of nonlinear systems from data, instead of equations.
△ Less
Submitted 30 March, 2023; v1 submitted 12 October, 2022;
originally announced October 2022.
-
Non-equilibrium Ensembles for the three-dimensional Navier-Stokes Equations
Authors:
Georgios Margazoglou,
Luca Biferale,
Massimo Cencini,
Giovanni Gallavotti,
Valerio Lucarini
Abstract:
At the molecular level fluid motions are, by first principles, described by time reversible laws. On the other hand, the coarse grained macroscopic evolution is suitably described by the Navier-Stokes equations, which are inherently irreversible, due to the dissipation term. Here, a reversible version of three-dimensional Navier-Stokes is studied, by introducing a fluctuating viscosity constructed…
▽ More
At the molecular level fluid motions are, by first principles, described by time reversible laws. On the other hand, the coarse grained macroscopic evolution is suitably described by the Navier-Stokes equations, which are inherently irreversible, due to the dissipation term. Here, a reversible version of three-dimensional Navier-Stokes is studied, by introducing a fluctuating viscosity constructed in such a way that enstrophy is conserved, along the lines of the paradigm of microcanonical versus canonical treatment in equilibrium statistical mechanics. Through systematic simulations we attack two important questions: (a) What are the conditions that must be satisfied in order to have a statistical equivalence between the two non-equilibrium ensembles? (b) What is the empirical distribution of the fluctuating viscosity observed by changing the Reynolds number and the number of modes used in the discretization of the evolution equation? The latter point is important also to establish regularity conditions for the reversible equations. We find that the probability to observe negative values of the fluctuating viscosity becomes very quickly extremely small when increasing the effective Reynolds number of the flow in the fully resolved hydro dynamical regime, at difference from what was observed previously.
△ Less
Submitted 23 June, 2022; v1 submitted 3 January, 2022;
originally announced January 2022.
-
Lévy-noise versus Gaussian-noise-induced Transitions in the Ghil-Sellers Energy Balance Model
Authors:
Valerio Lucarini,
Larissa Serdukova,
Georgios Margazoglou
Abstract:
We study the impact of applying stochastic forcing to the Ghil-Sellers energy balance climate model in the form of a fluctuating solar irradiance. Through numerical simulations, we explore the noise-induced transitions between the competing warm and snowball climate states. We consider multiplicative stochastic forcing driven by Gaussian and $α$-stable Lévy - $α\in(0,2)$ - noise laws, examine the…
▽ More
We study the impact of applying stochastic forcing to the Ghil-Sellers energy balance climate model in the form of a fluctuating solar irradiance. Through numerical simulations, we explore the noise-induced transitions between the competing warm and snowball climate states. We consider multiplicative stochastic forcing driven by Gaussian and $α$-stable Lévy - $α\in(0,2)$ - noise laws, examine the statistics of transition times, and estimate most probable transition paths. While the Gaussian noise case has been carefully studied in a plethora of investigations on metastable systems, much less is known about the Lévy case, especially in the case of high- and infinite-dimensional systems. In the weak noise limit, the expected residence time in each metastable state scales in a fundamentally different way in the Gaussian vs. Lévy noise case with respect to the intensity of the noise. In the former case, the classical Kramers-like exponential law is recovered. In the latter case, power laws are found, with the exponent equal to $-α$, in apparent agreement with rigorous results obtained for additive noise in a related - yet different - reaction-diffusion equation as well as in simpler models. This can be better understood by treating the Lévy noise as a compound Poisson process. The transition paths are studied in a projection of the state space and remarkable differences are observed between the two different types of noise. The snowball-to-warm and the warm-to-snowball most probable transition path cross at the single unstable edge state on the basin boundary. In the case of Lévy noise, the most probable transition paths in the two directions are wholly separated, as transitions apparently take place via the closest basin boundary region to the outgoing attractor. This property can be better elucidated by considering singular perturbations to the solar irradiance.
△ Less
Submitted 19 May, 2022; v1 submitted 20 October, 2021;
originally announced October 2021.
-
Dynamical Landscape and Multistability of a Climate Model
Authors:
Georgios Margazoglou,
Tobias Grafke,
Alessandro Laio,
Valerio Lucarini
Abstract:
We apply two independent data analysis methodologies to locate stable climate states in an intermediate complexity climate model and analyze their interplay. First, drawing from the theory of quasipotentials, and viewing the state space as an energy landscape with valleys and mountain ridges, we infer the relative likelihood of the identified multistable climate states, and investigate the most li…
▽ More
We apply two independent data analysis methodologies to locate stable climate states in an intermediate complexity climate model and analyze their interplay. First, drawing from the theory of quasipotentials, and viewing the state space as an energy landscape with valleys and mountain ridges, we infer the relative likelihood of the identified multistable climate states, and investigate the most likely transition trajectories as well as the expected transition times between them. Second, harnessing techniques from data science, specifically manifold learning, we characterize the data landscape of the simulation output to find climate states and basin boundaries within a fully agnostic and unsupervised framework. Both approaches show remarkable agreement, and reveal, apart from the well known warm and snowball earth states, a third intermediate stable state in one of the two climate models we consider. The combination of our approaches allows to identify how the negative feedback of ocean heat transport and entropy production via the hydrological cycle drastically change the topography of the dynamical landscape of Earth's climate.
△ Less
Submitted 8 January, 2021; v1 submitted 20 October, 2020;
originally announced October 2020.
-
Instanton based importance sampling for rare events in stochastic PDEs
Authors:
Lasse Ebener,
Georgios Margazoglou,
Jan Friedrich,
Luca Biferale,
Rainer Grauer
Abstract:
We present a new method for sampling rare and large fluctuations in a non-equilibrium system governed by a stochastic partial differential equation (SPDE) with additive forcing. To this end, we deploy the so-called instanton formalism that corresponds to a saddle-point approximation of the action in the path integral formulation of the underlying SPDE. The crucial step in our approach is the formu…
▽ More
We present a new method for sampling rare and large fluctuations in a non-equilibrium system governed by a stochastic partial differential equation (SPDE) with additive forcing. To this end, we deploy the so-called instanton formalism that corresponds to a saddle-point approximation of the action in the path integral formulation of the underlying SPDE. The crucial step in our approach is the formulation of an alternative SPDE that incorporates knowledge of the instanton solution such that we are able to constrain the dynamical evolutions around extreme flow configurations only. Finally, a reweighting procedure based on the Girsanov theorem is applied to recover the full distribution function of the original system. The entire procedure is demonstrated on the example of the one-dimensional Burgers equation. Furthermore, we compare our method to conventional direct numerical simulations as well as to Hybrid Monte Carlo methods. It will be shown that the instanton-based sampling method outperforms both approaches and allows for an accurate quantification of the whole probability density function of velocity gradients from the core to the very far tails.
△ Less
Submitted 9 December, 2018;
originally announced December 2018.
-
A Hybrid Monte Carlo algorithm for sampling rare events in space-time histories of stochastic fields
Authors:
G. Margazoglou,
L. Biferale,
R. Grauer,
K. Jansen,
D. Mesterházy,
T. Rosenow,
R. Tripiccione
Abstract:
We introduce a variant of the Hybrid Monte Carlo (HMC) algorithm to address large-deviation statistics in stochastic hydrodynamics. Based on the path-integral approach to stochastic (partial) differential equations, our HMC algorithm samples space-time histories of the dynamical degrees of freedom under the influence of random noise. First, we validate and benchmark the HMC algorithm by reproducin…
▽ More
We introduce a variant of the Hybrid Monte Carlo (HMC) algorithm to address large-deviation statistics in stochastic hydrodynamics. Based on the path-integral approach to stochastic (partial) differential equations, our HMC algorithm samples space-time histories of the dynamical degrees of freedom under the influence of random noise. First, we validate and benchmark the HMC algorithm by reproducing multiscale properties of the one-dimensional Burgers equation driven by Gaussian and white-in-time noise. Second, we show how to implement an importance sampling protocol to significantly enhance, by orders of magnitudes, the probability to sample extreme and rare events, making it possible to estimate moments of field variables of extremely high order (up to 30 and more). By employing reweighting techniques, we map the biased configurations back to the original probability measure in order to probe their statistical importance. Finally, we show that by biasing the system towards very intense negative gradients, the HMC algorithm is able to explore the statistical fluctuations around instanton configurations. Our results will also be interesting and relevant in lattice gauge theory since they provide insight into reweighting techniques.
△ Less
Submitted 27 October, 2019; v1 submitted 5 August, 2018;
originally announced August 2018.
-
Multiscale velocity correlations in turbulence and Burgers turbulence: Fusion rules, Markov processes in scale, and multifractal predictions
Authors:
Jan Friedrich,
Georgios Margazoglou,
Luca Biferale,
Rainer Grauer
Abstract:
We compare different approaches towards an effective description of multi-scale velocity field correlations in turbulence. Predictions made by the operator product expansion, the so-called fusion rules, are placed in juxtaposition to an approach that interprets the turbulent energy cascade in terms of a Markov process of velocity increments in scale. We explicitly show that the fusion rules are a…
▽ More
We compare different approaches towards an effective description of multi-scale velocity field correlations in turbulence. Predictions made by the operator product expansion, the so-called fusion rules, are placed in juxtaposition to an approach that interprets the turbulent energy cascade in terms of a Markov process of velocity increments in scale. We explicitly show that the fusion rules are a direct consequence of the Markov property provided that the structure functions exhibit scaling in the inertial range. Furthermore, the limit case of joint velocity gradient and velocity increment statistics is discussed and put into the context of the notion of dissipative anomaly. We generalize a prediction made by the multifractal (MF) approach derived in [Phys. Rev. Lett. 80, 3244 (1998)] to correlations among inertial range velocity increment and velocity gradients of any order. We show that for the case of squared velocity gradients such a relation can be derived from "first principles" in the case of Burgers equation. Our results are benchmarked by intensive direct numerical simulations of Burgers turbulence.
△ Less
Submitted 17 September, 2018; v1 submitted 24 April, 2018;
originally announced April 2018.
-
Domain magnetization approach to the isothermal critical exponent
Authors:
A. -M. Tsopelakou,
G. Margazoglou,
Y. F. Contoyiannis,
P. A. Kalozoumis,
F. K. Diakonos,
N. G. Fytas
Abstract:
We propose a method for calculating the isothermal critical exponent $δ$ in Ising systems undergoing a second-order phase transition. It is based on the calculation of the mean magnetization time series within a small connected domain of a lattice after equilibrium is reached. At the pseudocritical point, the magnetization time series attains intermittent characteristics and the probability densit…
▽ More
We propose a method for calculating the isothermal critical exponent $δ$ in Ising systems undergoing a second-order phase transition. It is based on the calculation of the mean magnetization time series within a small connected domain of a lattice after equilibrium is reached. At the pseudocritical point, the magnetization time series attains intermittent characteristics and the probability density for consecutive values of mean magnetization within a region around zero becomes a power law. Typically the size of this region is of the order of the standard deviation of the magnetization. The emerging power-law exponent is directly related to the isothermal critical exponent $δ$ through a simple analytical expression. We employ this method to calculate with remarkable accuracy the exponent $δ$ for the square-lattice Ising model where traditional approaches, like the constrained effective potential, typically fail to provide accurate results.
△ Less
Submitted 23 January, 2016;
originally announced January 2016.
