-
Neural General Circulation Models for Weather and Climate
Authors:
Dmitrii Kochkov,
Janni Yuval,
Ian Langmore,
Peter Norgaard,
Jamie Smith,
Griffin Mooers,
Milan Klöwer,
James Lottes,
Stephan Rasp,
Peter Düben,
Sam Hatfield,
Peter Battaglia,
Alvaro Sanchez-Gonzalez,
Matthew Willson,
Michael P. Brenner,
Stephan Hoyer
Abstract:
General circulation models (GCMs) are the foundation of weather and climate prediction. GCMs are physics-based simulators which combine a numerical solver for large-scale dynamics with tuned representations for small-scale processes such as cloud formation. Recently, machine learning (ML) models trained on reanalysis data achieved comparable or better skill than GCMs for deterministic weather fore…
▽ More
General circulation models (GCMs) are the foundation of weather and climate prediction. GCMs are physics-based simulators which combine a numerical solver for large-scale dynamics with tuned representations for small-scale processes such as cloud formation. Recently, machine learning (ML) models trained on reanalysis data achieved comparable or better skill than GCMs for deterministic weather forecasting. However, these models have not demonstrated improved ensemble forecasts, or shown sufficient stability for long-term weather and climate simulations. Here we present the first GCM that combines a differentiable solver for atmospheric dynamics with ML components, and show that it can generate forecasts of deterministic weather, ensemble weather and climate on par with the best ML and physics-based methods. NeuralGCM is competitive with ML models for 1-10 day forecasts, and with the European Centre for Medium-Range Weather Forecasts ensemble prediction for 1-15 day forecasts. With prescribed sea surface temperature, NeuralGCM can accurately track climate metrics such as global mean temperature for multiple decades, and climate forecasts with 140 km resolution exhibit emergent phenomena such as realistic frequency and trajectories of tropical cyclones. For both weather and climate, our approach offers orders of magnitude computational savings over conventional GCMs. Our results show that end-to-end deep learning is compatible with tasks performed by conventional GCMs, and can enhance the large-scale physical simulations that are essential for understanding and predicting the Earth system.
△ Less
Submitted 7 March, 2024; v1 submitted 13 November, 2023;
originally announced November 2023.
-
Recurrent flow patterns as a basis for turbulence: predicting statistics from structures
Authors:
Jacob Page,
Peter Norgaard,
Michael P. Brenner,
Rich R. Kerswell
Abstract:
A dynamical systems approach to turbulence envisions the flow as a trajectory through a high-dimensional state space transiently visiting the neighbourhoods of unstable simple invariant solutions (E. Hopf, Commun. Appl. Maths 1, 303, 1948). The hope has always been to turn this appealing picture into a predictive framework where the statistics of the flow follows from a weighted sum of the statist…
▽ More
A dynamical systems approach to turbulence envisions the flow as a trajectory through a high-dimensional state space transiently visiting the neighbourhoods of unstable simple invariant solutions (E. Hopf, Commun. Appl. Maths 1, 303, 1948). The hope has always been to turn this appealing picture into a predictive framework where the statistics of the flow follows from a weighted sum of the statistics of each simple invariant solution. Two outstanding obstacles have prevented this goal from being achieved: (1) paucity of known solutions and (2) the lack of a rational theory for predicting the required weights. Here we describe a method to substantially solve these problems, and thereby provide the first compelling evidence that the PDFs of a fully developed turbulent flow can be reconstructed with a set of unstable periodic orbits. Our new method for finding solutions uses automatic differentiation, with high-quality guesses constructed by minimising a trajectory-dependent loss function. We use this approach to find hundreds of new solutions in turbulent, two-dimensional Kolmogorov flow. Robust statistical predictions are then computed by learning weights after converting a turbulent trajectory into a Markov chain for which the states are individual solutions, and the nearest solution to a given snapshot is determined using a deep convolutional autoencoder. To our knowledge, this is the first time the PDFs of a spatio-temporally-chaotic system have been successfully reproduced with a set of simple invariant states, and provides a fascinating connection between self-sustaining dynamical processes and the more well-known statistical properties of turbulence.
△ Less
Submitted 14 November, 2023; v1 submitted 22 December, 2022;
originally announced December 2022.
-
Learning to correct spectral methods for simulating turbulent flows
Authors:
Gideon Dresdner,
Dmitrii Kochkov,
Peter Norgaard,
Leonardo Zepeda-Núñez,
Jamie A. Smith,
Michael P. Brenner,
Stephan Hoyer
Abstract:
Despite their ubiquity throughout science and engineering, only a handful of partial differential equations (PDEs) have analytical, or closed-form solutions. This motivates a vast amount of classical work on numerical simulation of PDEs and more recently, a whirlwind of research into data-driven techniques leveraging machine learning (ML). A recent line of work indicates that a hybrid of classical…
▽ More
Despite their ubiquity throughout science and engineering, only a handful of partial differential equations (PDEs) have analytical, or closed-form solutions. This motivates a vast amount of classical work on numerical simulation of PDEs and more recently, a whirlwind of research into data-driven techniques leveraging machine learning (ML). A recent line of work indicates that a hybrid of classical numerical techniques and machine learning can offer significant improvements over either approach alone. In this work, we show that the choice of the numerical scheme is crucial when incorporating physics-based priors. We build upon Fourier-based spectral methods, which are known to be more efficient than other numerical schemes for simulating PDEs with smooth and periodic solutions. Specifically, we develop ML-augmented spectral solvers for three common PDEs of fluid dynamics. Our models are more accurate (2-4x) than standard spectral solvers at the same resolution but have longer overall runtimes (~2x), due to the additional runtime cost of the neural network component. We also demonstrate a handful of key design principles for combining machine learning and numerical methods for solving PDEs.
△ Less
Submitted 25 June, 2023; v1 submitted 1 July, 2022;
originally announced July 2022.
-
Hamiltonian Monte Carlo in Inverse Problems; Ill-Conditioning and Multi-Modality
Authors:
Ian Langmore,
Michael Dikovsky,
Scott Geraedts,
Peter Norgaard,
Rob von Behren
Abstract:
The Hamiltonian Monte Carlo (HMC) method allows sampling from continuous densities. Favorable scaling with dimension has led to wide adoption of HMC by the statistics community. Modern auto-differentiating software should allow more widespread usage in Bayesian inverse problems. This paper analyzes two major difficulties encountered using HMC for inverse problems: poor conditioning and multi-modal…
▽ More
The Hamiltonian Monte Carlo (HMC) method allows sampling from continuous densities. Favorable scaling with dimension has led to wide adoption of HMC by the statistics community. Modern auto-differentiating software should allow more widespread usage in Bayesian inverse problems. This paper analyzes two major difficulties encountered using HMC for inverse problems: poor conditioning and multi-modality. Novel results on preconditioning and replica exchange Monte Carlo parameter selection are presented in the context of spectroscopy. Recommendations are given for the number of integration steps as well as step size, preconditioner type and fitting, annealing form and schedule. These recommendations are analyzed rigorously in the Gaussian case, and shown to generalize in a fusion plasma reconstruction.
△ Less
Submitted 15 August, 2022; v1 submitted 12 March, 2021;
originally announced March 2021.
-
A Condition Number for Hamiltonian Monte Carlo
Authors:
Ian Langmore,
Michael Dikovsky,
Scott Geraedts,
Peter Norgaard,
Rob Von Behren
Abstract:
Hamiltonian Monte Carlo is a popular sampling technique for smooth target densities. The scale lengths of the target have long been known to influence integration error and sampling efficiency. However, quantitative measures intrinsic to the target have been lacking. In this paper, we restrict attention to the multivariate Gaussian and the leapfrog integrator, and obtain a condition number corresp…
▽ More
Hamiltonian Monte Carlo is a popular sampling technique for smooth target densities. The scale lengths of the target have long been known to influence integration error and sampling efficiency. However, quantitative measures intrinsic to the target have been lacking. In this paper, we restrict attention to the multivariate Gaussian and the leapfrog integrator, and obtain a condition number corresponding to sampling efficiency. This number, based on the spectral and Schatten norms, quantifies the number of leapfrog steps needed to efficiently sample. We demonstrate its utility by using this condition number to analyze HMC preconditioning techniques. We also find the condition number of large inverse Wishart matrices, from which we derive burn-in heuristics.
△ Less
Submitted 4 February, 2020; v1 submitted 23 May, 2019;
originally announced May 2019.
