Tracking and forecasting oscillatory data streams using Koopman autoencoders and Kalman filtering
Authors:
Stephen A Falconer,
David J. B. Lloyd,
Naratip Santitissadeekorn
Abstract:
Data-driven modelling techniques provide a method for deriving models of dynamical systems directly from complicated data streams. However, tracking and forecasting such data streams poses a significant challenge to most methods, as they assume the underlying process and model does not change over time. In this paper, we apply one such data-driven method, the Koopman autoencoder (KAE), to high-dim…
▽ More
Data-driven modelling techniques provide a method for deriving models of dynamical systems directly from complicated data streams. However, tracking and forecasting such data streams poses a significant challenge to most methods, as they assume the underlying process and model does not change over time. In this paper, we apply one such data-driven method, the Koopman autoencoder (KAE), to high-dimensional oscillatory data to generate a low-dimensional latent space and model, where the system's dynamics appear linear. This allows one to accurately track and forecast systems where the underlying model may change over time. States and the model in the reduced order latent space can then be efficiently updated as new data becomes available, using data assimilation techniques such as the ensemble Kalman filter (EnKF), in a technique we call the KAE EnKF. We demonstrate that this approach is able to effectively track and forecast time-varying, nonlinear dynamical systems in synthetic examples. We then apply the KAE EnKF to a video of a physical pendulum, and achieve a significant improvement over current state-of-the-art methods. By generating effective latent space reconstructions, we find that we are able to construct accurate short-term forecasts and efficient adaptations to externally forced changes to the pendulum's frequency.
△ Less
Submitted 3 May, 2024;
originally announced May 2024.
Combining Dynamic Mode Decomposition with Ensemble Kalman Filtering for Tracking and Forecasting
Authors:
Stephen A Falconer,
David J. B. Lloyd,
Naratip Santitissadeekorn
Abstract:
Data assimilation techniques, such as ensemble Kalman filtering, have been shown to be a highly effective and efficient way to combine noisy data with a mathematical model to track and forecast dynamical systems. However, when dealing with high-dimensional data, in many situations one does not have a model, so data assimilation techniques cannot be applied. In this paper, we use dynamic mode decom…
▽ More
Data assimilation techniques, such as ensemble Kalman filtering, have been shown to be a highly effective and efficient way to combine noisy data with a mathematical model to track and forecast dynamical systems. However, when dealing with high-dimensional data, in many situations one does not have a model, so data assimilation techniques cannot be applied. In this paper, we use dynamic mode decomposition to generate a low-dimensional, linear model of a dynamical system directly from high-dimensional data, which is defined by temporal and spatial modes, that we can then use with data assimilation techniques such as the ensemble Kalman filter. We show how the dynamic mode decomposition can be combined with the ensemble Kalman filter (which we call the DMDEnKF) to iteratively update the current state and temporal modes as new data becomes available. We demonstrate that this approach is able to track time varying dynamical systems in synthetic examples, and experiment with the use of time-delay embeddings. We then apply the DMDEnKF to real world seasonal influenza-like illness data from the USA Centers for Disease Control and Prevention, and find that for short term forecasting, the DMDEnKF is comparable to the best mechanistic models in the ILINet competition.
△ Less
Submitted 13 January, 2023;
originally announced January 2023.
