-
Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems
Authors:
Ravi Chepuri,
Dael Amzalag,
Thomas Antonsen Jr.,
Michelle Girvan
Abstract:
Reservoir computers (RCs) are powerful machine learning architectures for time series prediction. Recently, next generation reservoir computers (NGRCs) have been introduced, offering distinct advantages over RCs, such as reduced computational expense and lower training data requirements. However, NGRCs have their own practical difficulties, including sensitivity to sampling time and type of nonlin…
▽ More
Reservoir computers (RCs) are powerful machine learning architectures for time series prediction. Recently, next generation reservoir computers (NGRCs) have been introduced, offering distinct advantages over RCs, such as reduced computational expense and lower training data requirements. However, NGRCs have their own practical difficulties, including sensitivity to sampling time and type of nonlinearities in the data. Here, we introduce a hybrid RC-NGRC approach for time series forecasting of dynamical systems. We show that our hybrid approach can produce accurate short term predictions and capture the long term statistics of chaotic dynamical systems in situations where the RC and NGRC components alone are insufficient, e.g., due to constraints from limited computational resources, sub-optimal hyperparameters, sparsely-sampled training data, etc. Under these conditions, we show for multiple model chaotic systems that the hybrid RC-NGRC method with a small reservoir can achieve prediction performance approaching that of a traditional RC with a much larger reservoir, illustrating that the hybrid approach can offer significant gains in computational efficiency over traditional RCs while simultaneously addressing some of the limitations of NGRCs. Our results suggest that hybrid RC-NGRC approach may be particularly beneficial in cases when computational efficiency is a high priority and an NGRC alone is not adequate.
△ Less
Submitted 5 June, 2024; v1 submitted 4 March, 2024;
originally announced March 2024.
-
Stabilizing Machine Learning Prediction of Dynamics: Noise and Noise-inspired Regularization
Authors:
Alexander Wikner,
Joseph Harvey,
Michelle Girvan,
Brian R. Hunt,
Andrew Pomerance,
Thomas Antonsen,
Edward Ott
Abstract:
Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model o…
▽ More
Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this technique can result in artificially rapid error growth. In this article, we systematically examine the technique of adding noise to the ML model input during training to promote stability and improve prediction accuracy. Furthermore, we introduce Linearized Multi-Noise Training (LMNT), a regularization technique that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. Our case study uses reservoir computing, a machine-learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while reservoir computers trained without regularization are unstable. Compared with other regularization techniques that yield stability in some cases, we find that both short-term and climate predictions from reservoir computers trained with noise or with LMNT are substantially more accurate. Finally, we show that the deterministic aspect of our LMNT regularization facilitates fast hyperparameter tuning when compared to training with noise.
△ Less
Submitted 12 December, 2022; v1 submitted 9 November, 2022;
originally announced November 2022.
-
Parallel Machine Learning for Forecasting the Dynamics of Complex Networks
Authors:
Keshav Srinivasan,
Nolan Coble,
Joy Hamlin,
Thomas Antonsen,
Edward Ott,
Michelle Girvan
Abstract:
Forecasting the dynamics of large complex networks from previous time-series data is important in a wide range of contexts. Here we present a machine learning scheme for this task using a parallel architecture that mimics the topology of the network of interest. We demonstrate the utility and scalability of our method implemented using reservoir computing on a chaotic network of oscillators. Two l…
▽ More
Forecasting the dynamics of large complex networks from previous time-series data is important in a wide range of contexts. Here we present a machine learning scheme for this task using a parallel architecture that mimics the topology of the network of interest. We demonstrate the utility and scalability of our method implemented using reservoir computing on a chaotic network of oscillators. Two levels of prior knowledge are considered: (i) the network links are known; and (ii) the network links are unknown and inferred via a data-driven approach to approximately optimize prediction.
△ Less
Submitted 27 August, 2021;
originally announced August 2021.
-
The Impact of Imperfect Information on Network Attack
Authors:
Andrew Melchionna,
Jesus Caloca,
Shane Squires,
Thomas M. Antonsen,
Edward Ott,
Michelle Girvan
Abstract:
This paper explores the effectiveness of network attack when the attacker has imperfect information about the network. For Erdős-Rényi networks, we observe that dynamical importance and betweenness centrality-based attacks are surprisingly robust to the presence of a moderate amount of imperfect information and are more effective compared with simpler degree-based attacks even at moderate levels o…
▽ More
This paper explores the effectiveness of network attack when the attacker has imperfect information about the network. For Erdős-Rényi networks, we observe that dynamical importance and betweenness centrality-based attacks are surprisingly robust to the presence of a moderate amount of imperfect information and are more effective compared with simpler degree-based attacks even at moderate levels of network information error. In contrast, for scale-free networks the effectiveness of attack is much less degraded by a moderate level of information error. Furthermore, in the Erdőos-Rényi case the effectiveness of network attack is much more degraded by missing links as compared with the same number of false links.
△ Less
Submitted 10 December, 2014;
originally announced December 2014.
-
Spatially embedded growing small-world networks
Authors:
Ari Zitin,
Alex Gorowora,
Shane Squires,
Mark Herrera,
Thomas M. Antonsen,
Michelle Girvan,
Edward Ott
Abstract:
Networks in nature are often formed within a spatial domain in a dynamical manner, gaining links and nodes as they develop over time. We propose a class of spatially-based growing network models and investigate the relationship between the resulting statistical network properties and the dimension and topology of the space in which the networks are embedded. In particular, we consider models in wh…
▽ More
Networks in nature are often formed within a spatial domain in a dynamical manner, gaining links and nodes as they develop over time. We propose a class of spatially-based growing network models and investigate the relationship between the resulting statistical network properties and the dimension and topology of the space in which the networks are embedded. In particular, we consider models in which nodes are placed one by one in random locations in space, with each such placement followed by configuration relaxation toward uniform node density, and connection of the new node with spatially nearby nodes. We find that such growth processes naturally result in networks with small-world features, including a short characteristic path length and nonzero clustering. These properties do not appear to depend strongly on the topology of the embedding space, but do depend strongly on its dimension; higher-dimensional spaces result in shorter path lengths but less clustering.
△ Less
Submitted 26 December, 2013;
originally announced December 2013.
