-
ArchesWeather: An efficient AI weather forecasting model at 1.5° resolution
Authors:
Guillaume Couairon,
Christian Lessig,
Anastase Charantonis,
Claire Monteleoni
Abstract:
One of the guiding principles for designing AI-based weather forecasting systems is to embed physical constraints as inductive priors in the neural network architecture. A popular prior is locality, where the atmospheric data is processed with local neural interactions, like 3D convolutions or 3D local attention windows as in Pangu-Weather. On the other hand, some works have shown great success in…
▽ More
One of the guiding principles for designing AI-based weather forecasting systems is to embed physical constraints as inductive priors in the neural network architecture. A popular prior is locality, where the atmospheric data is processed with local neural interactions, like 3D convolutions or 3D local attention windows as in Pangu-Weather. On the other hand, some works have shown great success in weather forecasting without this locality principle, at the cost of a much higher parameter count. In this paper, we show that the 3D local processing in Pangu-Weather is computationally sub-optimal. We design ArchesWeather, a transformer model that combines 2D attention with a column-wise attention-based feature interaction module, and demonstrate that this design improves forecasting skill.
ArchesWeather is trained at 1.5° resolution and 24h lead time, with a training budget of a few GPU-days and a lower inference cost than competing methods. An ensemble of four of our models shows better RMSE scores than the IFS HRES and is competitive with the 1.4° 50-members NeuralGCM ensemble for one to three days ahead forecasting. Our code and models are publicly available at https://github.com/gcouairon/ArchesWeather.
△ Less
Submitted 3 July, 2024; v1 submitted 23 May, 2024;
originally announced May 2024.
-
Towards a GPU-Parallelization of the neXtSIM-DG Dynamical Core
Authors:
Robert Jendersie,
Christian Lessig,
Thomas Richter
Abstract:
The cryosphere plays a significant role in Earth's climate system. Therefore, an accurate simulation of sea ice is of great importance to improve climate projections. To enable higher resolution simulations, graphics processing units (GPUs) have become increasingly attractive as they offer higher floating point peak performance and better energy efficiency compared to CPUs. However, making use of…
▽ More
The cryosphere plays a significant role in Earth's climate system. Therefore, an accurate simulation of sea ice is of great importance to improve climate projections. To enable higher resolution simulations, graphics processing units (GPUs) have become increasingly attractive as they offer higher floating point peak performance and better energy efficiency compared to CPUs. However, making use of this theoretical peak performance, which is based on massive data parallelism, usually requires more care and effort in the implementation. In recent years, a number of frameworks have become available that promise to simplify general purpose GPU programming. In this work, we compare multiple such frameworks, including CUDA, SYCL, Kokkos and PyTorch, for the parallelization of \nextsim, a finite-element based dynamical core for sea ice. We evaluate the different approaches according to their usability and performance.
△ Less
Submitted 28 February, 2024; v1 submitted 1 February, 2024;
originally announced February 2024.
-
AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learning
Authors:
Christian Lessig,
Ilaria Luise,
Bing Gong,
Michael Langguth,
Scarlet Stadtler,
Martin Schultz
Abstract:
The atmosphere affects humans in a multitude of ways, from loss of life due to adverse weather effects to long-term social and economic impacts on societies. Computer simulations of atmospheric dynamics are, therefore, of great importance for the well-being of our and future generations. Here, we propose AtmoRep, a novel, task-independent stochastic computer model of atmospheric dynamics that can…
▽ More
The atmosphere affects humans in a multitude of ways, from loss of life due to adverse weather effects to long-term social and economic impacts on societies. Computer simulations of atmospheric dynamics are, therefore, of great importance for the well-being of our and future generations. Here, we propose AtmoRep, a novel, task-independent stochastic computer model of atmospheric dynamics that can provide skillful results for a wide range of applications. AtmoRep uses large-scale representation learning from artificial intelligence to determine a general description of the highly complex, stochastic dynamics of the atmosphere from the best available estimate of the system's historical trajectory as constrained by observations. This is enabled by a novel self-supervised learning objective and a unique ensemble that samples from the stochastic model with a variability informed by the one in the historical record. The task-independent nature of AtmoRep enables skillful results for a diverse set of applications without specifically training for them and we demonstrate this for nowcasting, temporal interpolation, model correction, and counterfactuals. We also show that AtmoRep can be improved with additional data, for example radar observations, and that it can be extended to tasks such as downscaling. Our work establishes that large-scale neural networks can provide skillful, task-independent models of atmospheric dynamics. With this, they provide a novel means to make the large record of atmospheric observations accessible for applications and for scientific inquiry, complementing existing simulations based on first principles.
△ Less
Submitted 7 September, 2023; v1 submitted 25 August, 2023;
originally announced August 2023.
-
A Multi-Scale Deep Learning Framework for Projecting Weather Extremes
Authors:
Antoine Blanchard,
Nishant Parashar,
Boyko Dodov,
Christian Lessig,
Themistoklis Sapsis
Abstract:
Weather extremes are a major societal and economic hazard, claiming thousands of lives and causing billions of dollars in damage every year. Under climate change, their impact and intensity are expected to worsen significantly. Unfortunately, general circulation models (GCMs), which are currently the primary tool for climate projections, cannot characterize weather extremes accurately. To address…
▽ More
Weather extremes are a major societal and economic hazard, claiming thousands of lives and causing billions of dollars in damage every year. Under climate change, their impact and intensity are expected to worsen significantly. Unfortunately, general circulation models (GCMs), which are currently the primary tool for climate projections, cannot characterize weather extremes accurately. To address this, we present a multi-resolution deep-learning framework that, firstly, corrects a GCM's biases by matching low-order and tail statistics of its output with observations at coarse scales; and secondly, increases the level of detail of the debiased GCM output by reconstructing the finer scales as a function of the coarse scales. We use the proposed framework to generate statistically realistic realizations of the climate over Western Europe from a simple GCM corrected using observational atmospheric reanalysis. We also discuss implications for probabilistic risk assessment of natural disasters in a changing climate.
△ Less
Submitted 21 October, 2022;
originally announced October 2022.
-
AtmoDist: Self-supervised Representation Learning for Atmospheric Dynamics
Authors:
Sebastian Hoffmann,
Christian Lessig
Abstract:
Representation learning has proven to be a powerful methodology in a wide variety of machine learning applications. For atmospheric dynamics, however, it has so far not been considered, arguably due to the lack of large-scale, labeled datasets that could be used for training. In this work, we show that the difficulty is benign and introduce a self-supervised learning task that defines a categorica…
▽ More
Representation learning has proven to be a powerful methodology in a wide variety of machine learning applications. For atmospheric dynamics, however, it has so far not been considered, arguably due to the lack of large-scale, labeled datasets that could be used for training. In this work, we show that the difficulty is benign and introduce a self-supervised learning task that defines a categorical loss for a wide variety of unlabeled atmospheric datasets. Specifically, we train a neural network on the simple yet intricate task of predicting the temporal distance between atmospheric fields from distinct but nearby times. We demonstrate that training with this task on ERA5 reanalysis leads to internal representations capturing intrinsic aspects of atmospheric dynamics. We do so by introducing a data-driven distance metric for atmospheric states. When employed as a loss function in other machine learning applications, this Atmodist distance leads to improved results compared to the classical $\ell_2$-loss. For example, for downscaling one obtains higher resolution fields that match the true statistics more closely than previous approaches and for the interpolation of missing or occluded data the AtmoDist distance leads to results that contain more realistic fine scale features. Since it is derived from observational data, AtmoDist also provides a novel perspective on atmospheric predictability.
△ Less
Submitted 23 August, 2022; v1 submitted 2 February, 2022;
originally announced February 2022.
-
Variational symplectic diagonally implicit Runge-Kutta methods for isospectral systems
Authors:
Clauson Carvalho da Silva,
Christian Lessig
Abstract:
Isospectral flows appear in a variety of applications, e.g. the Toda lattice in solid state physics or in discrete models for two-dimensional hydrodynamics, with the isospectral property often corresponding to mathematically or physically important conservation laws. Their most prominent feature, i.e. the conservation of the eigenvalues of the matrix state variable, should therefore be retained wh…
▽ More
Isospectral flows appear in a variety of applications, e.g. the Toda lattice in solid state physics or in discrete models for two-dimensional hydrodynamics, with the isospectral property often corresponding to mathematically or physically important conservation laws. Their most prominent feature, i.e. the conservation of the eigenvalues of the matrix state variable, should therefore be retained when discretizing these systems. Recently, it was shown how isospectral Runge-Kutta methods can, in the Lie-Poisson case also considered in our work, be obtained through Hamiltonian reduction of symplectic Runge-Kutta methods on the cotangent bundle of a Lie group. We provide the Lagrangian analogue and, in the case of symplectic diagonal implicit Runge-Kutta methods, derive the methods through a discrete Euler-Poincare reduction. Our derivation relies on a formulation of diagonally implicit isospectral Runge-Kutta methods in terms of the Cayley transform, generalizing earlier work that showed this for the implicit midpoint rule. Our work is also a generalization of earlier variational Lie group integrators that, interestingly, appear when these are interpreted as update equations for intermediate time points. From a practical point of view, our results allow for a simple implementation of higher order isospectral methods and we demonstrate this with numerical experiments where both the isospectral property and energy are conserved to high accuracy.
△ Less
Submitted 27 December, 2021;
originally announced December 2021.
-
Towards Representation Learning for Atmospheric Dynamics
Authors:
Sebastian Hoffmann,
Christian Lessig
Abstract:
The prediction of future climate scenarios under anthropogenic forcing is critical to understand climate change and to assess the impact of potentially counter-acting technologies. Machine learning and hybrid techniques for this prediction rely on informative metrics that are sensitive to pertinent but often subtle influences. For atmospheric dynamics, a critical part of the climate system, no wel…
▽ More
The prediction of future climate scenarios under anthropogenic forcing is critical to understand climate change and to assess the impact of potentially counter-acting technologies. Machine learning and hybrid techniques for this prediction rely on informative metrics that are sensitive to pertinent but often subtle influences. For atmospheric dynamics, a critical part of the climate system, no well established metric exists and visual inspection is currently still often used in practice. However, this "eyeball metric" cannot be used for machine learning where an algorithmic description is required. Motivated by the success of intermediate neural network activations as basis for learned metrics, e.g. in computer vision, we present a novel, self-supervised representation learning approach specifically designed for atmospheric dynamics. Our approach, called AtmoDist, trains a neural network on a simple, auxiliary task: predicting the temporal distance between elements of a randomly shuffled sequence of atmospheric fields (e.g. the components of the wind field from reanalysis or simulation). The task forces the network to learn important intrinsic aspects of the data as activations in its layers and from these hence a discriminative metric can be obtained. We demonstrate this by using AtmoDist to define a metric for GAN-based super resolution of vorticity and divergence. Our upscaled data matches both visually and in terms of its statistics a high resolution reference closely and it significantly outperform the state-of-the-art based on mean squared error. Since AtmoDist is unsupervised, only requires a temporal sequence of fields, and uses a simple auxiliary task, it has the potential to be of utility in a wide range of applications.
△ Less
Submitted 30 November, 2021; v1 submitted 19 September, 2021;
originally announced September 2021.
-
Local Fourier Slice Photography
Authors:
Christian Lessig
Abstract:
Light field cameras provide intriguing possibilities, such as post-capture refocus or the ability to synthesize images from novel viewpoints. This comes, however, at the price of significant storage requirements. Compression techniques can be used to reduce these but refocusing and reconstruction require so far again a dense pixel representation. To avoid this, we introduce local Fourier slice pho…
▽ More
Light field cameras provide intriguing possibilities, such as post-capture refocus or the ability to synthesize images from novel viewpoints. This comes, however, at the price of significant storage requirements. Compression techniques can be used to reduce these but refocusing and reconstruction require so far again a dense pixel representation. To avoid this, we introduce local Fourier slice photography that allows for refocused image reconstruction directly from a sparse wavelet representation of a light field, either to obtain an image or a compressed representation of it. The result is made possible by wavelets that respect the "slicing's" intrinsic structure and enable us to derive exact reconstruction filters for the refocused image in closed form. Image reconstruction then amounts to applying these filters to the light field's wavelet coefficients, and hence no reconstruction of a dense pixel representation is required. We demonstrate that this substantially reduces storage requirements and also computation times. We furthermore analyze the computational complexity of our algorithm and show that it scales linearly with the size of the reconstructed region and the non-negligible wavelet coefficients, i.e. with the visual complexity.
△ Less
Submitted 10 October, 2019; v1 submitted 16 February, 2019;
originally announced February 2019.
-
Divergence Free Polar Wavelets for the Analysis and Representation of Fluid Flows
Authors:
Christian Lessig
Abstract:
We present a Parseval tight wavelet frame for the representation and analysis of velocity vector fields of incompressible fluids. Our wavelets have closed form expressions in the frequency and spatial domains, are divergence free in the ideal, analytic sense, have a multi-resolution structure and fast transforms, and an intuitive correspondence to common flow phenomena. Our construction also allow…
▽ More
We present a Parseval tight wavelet frame for the representation and analysis of velocity vector fields of incompressible fluids. Our wavelets have closed form expressions in the frequency and spatial domains, are divergence free in the ideal, analytic sense, have a multi-resolution structure and fast transforms, and an intuitive correspondence to common flow phenomena. Our construction also allows for well defined directional selectivity, e.g. to model the behavior of divergence free vector fields in the vicinity of boundaries or to represent highly directional features like in a von Kármán vortex street. We demonstrate the practicality and efficiency of our construction by analyzing the representation of different divergence free vector fields in our wavelets.
△ Less
Submitted 30 September, 2018; v1 submitted 5 May, 2018;
originally announced May 2018.
-
Polar Wavelets in Space
Authors:
Christian Lessig
Abstract:
Recent work introduced a unified framework for steerable and directional wavelets in two and three dimensions that ensures many desirable properties, such as a multi-scale structure, fast transforms, and a flexible angular localization. We show that, for an appropriate choice for the radial window function, these wavelets also have closed form expressions for, among other things, the spatial repre…
▽ More
Recent work introduced a unified framework for steerable and directional wavelets in two and three dimensions that ensures many desirable properties, such as a multi-scale structure, fast transforms, and a flexible angular localization. We show that, for an appropriate choice for the radial window function, these wavelets also have closed form expressions for, among other things, the spatial representation, the filter taps for the fast transform, and the frame representation of the Laplace operator. The numerical practicality and benefits of our work are demonstrated using signal estimation from non-uniform, point-wise samples, as required for example in ray tracing, and for reconstructing a signal over a lower-dimensional sub-manifold, with applications for instance in medical imaging.
△ Less
Submitted 5 May, 2018;
originally announced May 2018.
