-
A Generative Adversarial Network for Climate Tip** Point Discovery (TIP-GAN)
Authors:
Jennifer Sleeman,
David Chung,
Anand Gnanadesikan,
Jay Brett,
Yannis Kevrekidis,
Marisa Hughes,
Thomas Haine,
Marie-Aude Pradal,
Renske Gelderloos,
Chace Ashcraft,
Caroline Tang,
Anshu Saksena,
Larry White
Abstract:
We propose a new Tip** Point Generative Adversarial Network (TIP-GAN) for better characterizing potential climate tip** points in Earth system models. We describe an adversarial game to explore the parameter space of these models, detect upcoming tip** points, and discover the drivers of tip** points. In this setup, a set of generators learn to construct model configurations that will invo…
▽ More
We propose a new Tip** Point Generative Adversarial Network (TIP-GAN) for better characterizing potential climate tip** points in Earth system models. We describe an adversarial game to explore the parameter space of these models, detect upcoming tip** points, and discover the drivers of tip** points. In this setup, a set of generators learn to construct model configurations that will invoke a climate tip** point. The discriminator learns to identify which generators are generating each model configuration and whether a given configuration will lead to a tip** point. The discriminator is trained using an oracle (a surrogate climate model) to test if a generated model configuration leads to a tip** point or not. We demonstrate the application of this GAN to invoke the collapse of the Atlantic Meridional Overturning Circulation (AMOC). We share experimental results of modifying the loss functions and the number of generators to exploit the area of uncertainty in model state space near a climate tip** point. In addition, we show that our trained discriminator can predict AMOC collapse with a high degree of accuracy without the use of the oracle. This approach could generalize to other tip** points, and could augment climate modeling research by directing users interested in studying tip** points to parameter sets likely to induce said tip** points in their computationally intensive climate models.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
Using Artificial Intelligence to aid Scientific Discovery of Climate Tip** Points
Authors:
Jennifer Sleeman,
David Chung,
Chace Ashcraft,
Jay Brett,
Anand Gnanadesikan,
Yannis Kevrekidis,
Marisa Hughes,
Thomas Haine,
Marie-Aude Pradal,
Renske Gelderloos,
Caroline Tang,
Anshu Saksena,
Larry White
Abstract:
We propose a hybrid Artificial Intelligence (AI) climate modeling approach that enables climate modelers in scientific discovery using a climate-targeted simulation methodology based on a novel combination of deep neural networks and mathematical methods for modeling dynamical systems. The simulations are grounded by a neuro-symbolic language that both enables question answering of what is learned…
▽ More
We propose a hybrid Artificial Intelligence (AI) climate modeling approach that enables climate modelers in scientific discovery using a climate-targeted simulation methodology based on a novel combination of deep neural networks and mathematical methods for modeling dynamical systems. The simulations are grounded by a neuro-symbolic language that both enables question answering of what is learned by the AI methods and provides a means of explainability. We describe how this methodology can be applied to the discovery of climate tip** points and, in particular, the collapse of the Atlantic Meridional Overturning Circulation (AMOC). We show how this methodology is able to predict AMOC collapse with a high degree of accuracy using a surrogate climate model for ocean interaction. We also show preliminary results of neuro-symbolic method performance when translating between natural language questions and symbolically learned representations. Our AI methodology shows promising early results, potentially enabling faster climate tip** point related research that would otherwise be computationally infeasible.
△ Less
Submitted 14 February, 2023;
originally announced February 2023.
-
Benchmarking optimality of time series classification methods in distinguishing diffusions
Authors:
Zehong Zhang,
Fei Lu,
Esther Xu Fei,
Terry Lyons,
Yannis Kevrekidis,
Tom Woolf
Abstract:
Statistical optimality benchmarking is crucial for analyzing and designing time series classification (TSC) algorithms. This study proposes to benchmark the optimality of TSC algorithms in distinguishing diffusion processes by the likelihood ratio test (LRT). The LRT is an optimal classifier by the Neyman-Pearson lemma. The LRT benchmarks are computationally efficient because the LRT does not need…
▽ More
Statistical optimality benchmarking is crucial for analyzing and designing time series classification (TSC) algorithms. This study proposes to benchmark the optimality of TSC algorithms in distinguishing diffusion processes by the likelihood ratio test (LRT). The LRT is an optimal classifier by the Neyman-Pearson lemma. The LRT benchmarks are computationally efficient because the LRT does not need training, and the diffusion processes can be efficiently simulated and are flexible to reflect the specific features of real-world applications. We demonstrate the benchmarking with three widely-used TSC algorithms: random forest, ResNet, and ROCKET. These algorithms can achieve the LRT optimality for univariate time series and multivariate Gaussian processes. However, these model-agnostic algorithms are suboptimal in classifying high-dimensional nonlinear multivariate time series. Additionally, the LRT benchmark provides tools to analyze the dependence of classification accuracy on the time length, dimension, temporal sampling frequency, and randomness of the time series.
△ Less
Submitted 11 April, 2023; v1 submitted 30 January, 2023;
originally announced January 2023.
-
Unsupervised learning of observation functions in state-space models by nonparametric moment methods
Authors:
Qingci An,
Yannis Kevrekidis,
Fei Lu,
Mauro Maggioni
Abstract:
We investigate the unsupervised learning of non-invertible observation functions in nonlinear state-space models. Assuming abundant data of the observation process along with the distribution of the state process, we introduce a nonparametric generalized moment method to estimate the observation function via constrained regression. The major challenge comes from the non-invertibility of the observ…
▽ More
We investigate the unsupervised learning of non-invertible observation functions in nonlinear state-space models. Assuming abundant data of the observation process along with the distribution of the state process, we introduce a nonparametric generalized moment method to estimate the observation function via constrained regression. The major challenge comes from the non-invertibility of the observation function and the lack of data pairs between the state and observation. We address the fundamental issue of identifiability from quadratic loss functionals and show that the function space of identifiability is the closure of a RKHS that is intrinsic to the state process. Numerical results show that the first two moments and temporal correlations, along with upper and lower bounds, can identify functions ranging from piecewise polynomials to smooth functions, leading to convergent estimators. The limitations of this method, such as non-identifiability due to symmetry and stationarity, are also discussed.
△ Less
Submitted 11 July, 2022;
originally announced July 2022.
-
Applications of Koopman Mode Analysis to Neural Networks
Authors:
Iva Manojlović,
Maria Fonoberova,
Ryan Mohr,
Aleksandr Andrejčuk,
Zlatko Drmač,
Yannis Kevrekidis,
Igor Mezić
Abstract:
We consider the training process of a neural network as a dynamical system acting on the high-dimensional weight space. Each epoch is an application of the map induced by the optimization algorithm and the loss function. Using this induced map, we can apply observables on the weight space and measure their evolution. The evolution of the observables are given by the Koopman operator associated wit…
▽ More
We consider the training process of a neural network as a dynamical system acting on the high-dimensional weight space. Each epoch is an application of the map induced by the optimization algorithm and the loss function. Using this induced map, we can apply observables on the weight space and measure their evolution. The evolution of the observables are given by the Koopman operator associated with the induced dynamical system. We use the spectrum and modes of the Koopman operator to realize the above objectives. Our methods can help to, a priori, determine the network depth; determine if we have a bad initialization of the network weights, allowing a restart before training too long; speeding up the training time. Additionally, our methods help enable noise rejection and improve robustness. We show how the Koopman spectrum can be used to determine the number of layers required for the architecture. Additionally, we show how we can elucidate the convergence versus non-convergence of the training process by monitoring the spectrum, in particular, how the existence of eigenvalues clustering around 1 determines when to terminate the learning process. We also show how using Koopman modes we can selectively prune the network to speed up the training procedure. Finally, we show that incorporating loss functions based on negative Sobolev norms can allow for the reconstruction of a multi-scale signal polluted by very large amounts of noise.
△ Less
Submitted 21 June, 2020;
originally announced June 2020.
