-
An adaptive finite element multigrid solver using GPU acceleration
Authors:
Manuel Liebchen,
Utku Kaya,
Christian Lessig,
Thomas Richter
Abstract:
Adaptive finite elements combined with geometric multigrid solvers are one of the most efficient numerical methods for problems such as the instationary Navier-Stokes equations. Yet despite their efficiency, computations remain expensive and the simulation of, for example, complex flow problems can take many hours or days. GPUs provide an interesting avenue to speed up the calculations due to thei…
▽ More
Adaptive finite elements combined with geometric multigrid solvers are one of the most efficient numerical methods for problems such as the instationary Navier-Stokes equations. Yet despite their efficiency, computations remain expensive and the simulation of, for example, complex flow problems can take many hours or days. GPUs provide an interesting avenue to speed up the calculations due to their very large theoretical peak performance. However, the large degree of parallelism and non-standard API make the use of GPUs in scientific computing challenging. In this work, we develop a GPU acceleration for the adaptive finite element library Gascoigne and study its effectiveness for different systems of partial differential equations. Through the systematic formulation of all computations as linear algebra operations, we can employ GPU-accelerated linear algebra libraries, which simplifies the implementation and ensures the maintainability of the code while achieving very efficient GPU utilizations. Our results for a transport-diffusion equation, linear elasticity, and the instationary Navier-Stokes equations show substantial speedups of up to 20X compared to multi-core CPU implementations.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
Towards a GPU-Parallelization of the neXtSIM-DG Dynamical Core
Authors:
Robert Jendersie,
Christian Lessig,
Thomas Richter
Abstract:
The cryosphere plays a significant role in Earth's climate system. Therefore, an accurate simulation of sea ice is of great importance to improve climate projections. To enable higher resolution simulations, graphics processing units (GPUs) have become increasingly attractive as they offer higher floating point peak performance and better energy efficiency compared to CPUs. However, making use of…
▽ More
The cryosphere plays a significant role in Earth's climate system. Therefore, an accurate simulation of sea ice is of great importance to improve climate projections. To enable higher resolution simulations, graphics processing units (GPUs) have become increasingly attractive as they offer higher floating point peak performance and better energy efficiency compared to CPUs. However, making use of this theoretical peak performance, which is based on massive data parallelism, usually requires more care and effort in the implementation. In recent years, a number of frameworks have become available that promise to simplify general purpose GPU programming. In this work, we compare multiple such frameworks, including CUDA, SYCL, Kokkos and PyTorch, for the parallelization of \nextsim, a finite-element based dynamical core for sea ice. We evaluate the different approaches according to their usability and performance.
△ Less
Submitted 28 February, 2024; v1 submitted 1 February, 2024;
originally announced February 2024.
-
DNN-MG: A Hybrid Neural Network/Finite Element Method with Applications to 3D Simulations of the Navier-Stokes Equations
Authors:
Nils Margenberg,
Robert Jendersie,
Christian Lessig,
Thomas Richter
Abstract:
We extend and analyze the deep neural network multigrid solver (DNN-MG) for the Navier-Stokes equations in three dimensions. The idea of the method is to augment a finite element simulation on coarse grids with fine scale information obtained using deep neural networks.
The neural network operates locally on small patches of grid elements. The local approach proves to be highly efficient, since…
▽ More
We extend and analyze the deep neural network multigrid solver (DNN-MG) for the Navier-Stokes equations in three dimensions. The idea of the method is to augment a finite element simulation on coarse grids with fine scale information obtained using deep neural networks.
The neural network operates locally on small patches of grid elements. The local approach proves to be highly efficient, since the network can be kept (relatively) small and since it can be applied in parallel on all grid patches. However, the main advantage of the local approach is the inherent generalizability of the method. Since the network only processes data of small sub-areas, it never ``sees'' the global problem and thus does not learn false biases.
We describe the method with a focus on the interplay between the finite element method and deep neural networks. Further, we demonstrate with numerical examples the excellent efficiency of the hybrid approach, which allows us to achieve very high accuracy with a coarse grid and thus reduce the computation time by orders of magnitude.
△ Less
Submitted 1 November, 2023; v1 submitted 27 July, 2023;
originally announced July 2023.
-
Variational symplectic diagonally implicit Runge-Kutta methods for isospectral systems
Authors:
Clauson Carvalho da Silva,
Christian Lessig
Abstract:
Isospectral flows appear in a variety of applications, e.g. the Toda lattice in solid state physics or in discrete models for two-dimensional hydrodynamics, with the isospectral property often corresponding to mathematically or physically important conservation laws. Their most prominent feature, i.e. the conservation of the eigenvalues of the matrix state variable, should therefore be retained wh…
▽ More
Isospectral flows appear in a variety of applications, e.g. the Toda lattice in solid state physics or in discrete models for two-dimensional hydrodynamics, with the isospectral property often corresponding to mathematically or physically important conservation laws. Their most prominent feature, i.e. the conservation of the eigenvalues of the matrix state variable, should therefore be retained when discretizing these systems. Recently, it was shown how isospectral Runge-Kutta methods can, in the Lie-Poisson case also considered in our work, be obtained through Hamiltonian reduction of symplectic Runge-Kutta methods on the cotangent bundle of a Lie group. We provide the Lagrangian analogue and, in the case of symplectic diagonal implicit Runge-Kutta methods, derive the methods through a discrete Euler-Poincare reduction. Our derivation relies on a formulation of diagonally implicit isospectral Runge-Kutta methods in terms of the Cayley transform, generalizing earlier work that showed this for the implicit midpoint rule. Our work is also a generalization of earlier variational Lie group integrators that, interestingly, appear when these are interpreted as update equations for intermediate time points. From a practical point of view, our results allow for a simple implementation of higher order isospectral methods and we demonstrate this with numerical experiments where both the isospectral property and energy are conserved to high accuracy.
△ Less
Submitted 27 December, 2021;
originally announced December 2021.
-
Deep neural networks for geometric multigrid methods
Authors:
Nils Margenberg,
Robert Jendersie,
Thomas Richter,
Christian Lessig
Abstract:
We investigate scaling and efficiency of the deep neural network multigrid method (DNN-MG).
DNN-MG is a novel neural network-based technique for the simulation of the Navier-Stokes equations that combines an adaptive geometric multigrid solver, i.e. a highly efficient
classical solution scheme, with a recurrent neural network with memory.
The neural network replaces in DNN-MG one or multiple…
▽ More
We investigate scaling and efficiency of the deep neural network multigrid method (DNN-MG).
DNN-MG is a novel neural network-based technique for the simulation of the Navier-Stokes equations that combines an adaptive geometric multigrid solver, i.e. a highly efficient
classical solution scheme, with a recurrent neural network with memory.
The neural network replaces in DNN-MG one or multiple finest multigrid layers and provides a correction for the classical solve in the next time step.
This leads to little degradation in the solution quality while substantially reducing the overall computational costs.
At the same time, the use of the multigrid solver at the coarse scales allows for a compact network that is easy to train, generalizes well, and allows for the incorporation of physical constraints.
Previous work on DNN-MG focused on the overall scheme and how to enforce divergence freedom in the solution.
In this work, we investigate how the network size affects training and solution quality and the overall runtime of the computations.
Our results demonstrate that larger networks are able to capture the
flow behavior better while requiring only little additional training time.
At runtime, the use of the neural network correction can even reduce the computation time compared to a classical multigrid simulation through a faster convergence of the nonlinear solve that is required at every time step.
△ Less
Submitted 14 June, 2021;
originally announced June 2021.
-
Structure Preservation for the Deep Neural Network Multigrid Solver
Authors:
Nils Margenberg,
Christian Lessig,
Thomas Richter
Abstract:
The simulation of partial differential equations is a central subject of numerical analysis and an indispensable tool in science, engineering and related fields. Existing approaches, such as finite elements, provide (highly) efficient tools but deep neural network-based techniques emerged in the last few years as an alternative with very promising results. We investigate the combination of both ap…
▽ More
The simulation of partial differential equations is a central subject of numerical analysis and an indispensable tool in science, engineering and related fields. Existing approaches, such as finite elements, provide (highly) efficient tools but deep neural network-based techniques emerged in the last few years as an alternative with very promising results. We investigate the combination of both approaches for the approximation of the Navier-Stokes equations and to what extent structural properties such as divergence freedom can and should be respected. Our work is based on DNN-MG, a deep neural network multigrid technique, that we introduced recently and which uses a neural network to represent fine grid fluctuations not resolved by a geometric multigrid finite element solver. Although DNN-MG provides solutions with very good accuracy and is computationally highly efficient, we noticed that the neural network-based corrections substantially violate the divergence freedom of the velocity vector field. In this contribution, we discuss these findings and analyze three approaches to address the problem: a penalty term to encourage divergence freedom of the network output; a penalty term for the corrected velocity field; and a network that learns the stream function, i.e. the scalar potential of the divergence free velocity vector field and which hence yields by construction divergence free corrections. Our experimental results show that the third approach based on the stream function outperforms the other two and not only improves the divergence freedom but in particular also the overall fidelity of the simulation.
△ Less
Submitted 9 December, 2020;
originally announced December 2020.
-
A neural network multigrid solver for the Navier-Stokes equations
Authors:
Nils Margenberg,
Dirk Hartmann,
Christian Lessig,
Thomas Richter
Abstract:
We present the deep neural network multigrid solver (DNN-MG) that we develop for the instationary Navier-Stokes equations. DNN-MG improves computational efficiency using a judicious combination of a geometric multigrid solver and a recurrent neural network with memory. DNN-MG uses the multi-grid method to classically solve on coarse levels while the neural network corrects interpolated solutions o…
▽ More
We present the deep neural network multigrid solver (DNN-MG) that we develop for the instationary Navier-Stokes equations. DNN-MG improves computational efficiency using a judicious combination of a geometric multigrid solver and a recurrent neural network with memory. DNN-MG uses the multi-grid method to classically solve on coarse levels while the neural network corrects interpolated solutions on fine ones, thus avoiding the increasingly expensive computations that would have to be performed there. This results in a reduction in computation time through DNN-MG's highly compact neural network. The compactness results from its design for local patches and the available coarse multigrid solutions that provides a "guide" for the corrections. A compact neural network with a small number of parameters also reduces training time and data. Furthermore, the network's locality facilitates generalizability and allows one to use DNN-MG trained on one mesh domain also on different ones. We demonstrate the efficacy of DNN-MG for variations of the 2D laminar flow around an obstacle. For these, our method significantly improves the solutions as well as lift and drag functionals while requiring only about half the computation time of a full multigrid solution. We also show that DNN-MG trained for the configuration with one obstacle can be generalized to other time dependent problems that can be solved efficiently using a geometric multigrid method.
△ Less
Submitted 29 November, 2021; v1 submitted 26 August, 2020;
originally announced August 2020.
-
A Local Spectral Exterior Calculus for the Sphere and Application to the Shallow Water Equations
Authors:
Clauson Carvalho da Silva,
Christian Lessig,
Boyko Dodov,
Henk Dijkstra,
Themis Sapsis
Abstract:
We introduce $Ψ\mathrm{ec}$, a local spectral exterior calculus for the two-sphere $S^2$. $Ψ\mathrm{ec}$ provides a discretization of Cartan's exterior calculus on $S^2$ formed by spherical differential $r$-form wavelets. These are well localized in space and frequency and provide (Stevenson) frames for the homogeneous Sobolev spaces $\dot{H}^{-r+1}( Ω_ν^{r} , S^2 )$ of differential $r$-forms. At…
▽ More
We introduce $Ψ\mathrm{ec}$, a local spectral exterior calculus for the two-sphere $S^2$. $Ψ\mathrm{ec}$ provides a discretization of Cartan's exterior calculus on $S^2$ formed by spherical differential $r$-form wavelets. These are well localized in space and frequency and provide (Stevenson) frames for the homogeneous Sobolev spaces $\dot{H}^{-r+1}( Ω_ν^{r} , S^2 )$ of differential $r$-forms. At the same time, they satisfy important properties of the exterior calculus, such as the de Rahm complex and the Hodge-Helmholtz decomposition. Through this, $Ψ\mathrm{ec}$ is tailored towards structure preserving discretizations that can adapt to solutions with varying regularity. The construction of $Ψ\mathrm{ec}$ is based on a novel spherical wavelet frame for $L_2(S^2)$ that we obtain by introducing scalable reproducing kernel frames. These extend scalable frames to weighted sampling expansions and provide an alternative to quadrature rules for the discretization of needlet-like scale-discrete wavelets. We verify the practicality of $Ψ\mathrm{ec}$ for numerical computations using the rotating shallow water equations. Our numerical results demonstrate that a $Ψ\mathrm{ec}$-based discretization of the equations attains accuracy comparable to those of spectral methods while using a representation that is well localized in space and frequency.
△ Less
Submitted 7 May, 2020;
originally announced May 2020.
-
Local Fourier Slice Photography
Authors:
Christian Lessig
Abstract:
Light field cameras provide intriguing possibilities, such as post-capture refocus or the ability to synthesize images from novel viewpoints. This comes, however, at the price of significant storage requirements. Compression techniques can be used to reduce these but refocusing and reconstruction require so far again a dense pixel representation. To avoid this, we introduce local Fourier slice pho…
▽ More
Light field cameras provide intriguing possibilities, such as post-capture refocus or the ability to synthesize images from novel viewpoints. This comes, however, at the price of significant storage requirements. Compression techniques can be used to reduce these but refocusing and reconstruction require so far again a dense pixel representation. To avoid this, we introduce local Fourier slice photography that allows for refocused image reconstruction directly from a sparse wavelet representation of a light field, either to obtain an image or a compressed representation of it. The result is made possible by wavelets that respect the "slicing's" intrinsic structure and enable us to derive exact reconstruction filters for the refocused image in closed form. Image reconstruction then amounts to applying these filters to the light field's wavelet coefficients, and hence no reconstruction of a dense pixel representation is required. We demonstrate that this substantially reduces storage requirements and also computation times. We furthermore analyze the computational complexity of our algorithm and show that it scales linearly with the size of the reconstructed region and the non-negligible wavelet coefficients, i.e. with the visual complexity.
△ Less
Submitted 10 October, 2019; v1 submitted 16 February, 2019;
originally announced February 2019.
-
$Ψ$ec: A Local Spectral Exterior Calculus
Authors:
Christian Lessig
Abstract:
We introduce $Ψ\mathrm{ec}$, a discretization of Cartan's exterior calculus of differential forms using wavelets. Our construction consists of differential $r$-form wavelets with flexible directional localization that provide tight frames for the spaces $Ω^r(\mathbb{R}^n)$ of forms in $\mathbb{R}^2$ and $\mathbb{R}^3$. By construction, the wavelets satisfy the de Rahm co-chain complex, the Hodge d…
▽ More
We introduce $Ψ\mathrm{ec}$, a discretization of Cartan's exterior calculus of differential forms using wavelets. Our construction consists of differential $r$-form wavelets with flexible directional localization that provide tight frames for the spaces $Ω^r(\mathbb{R}^n)$ of forms in $\mathbb{R}^2$ and $\mathbb{R}^3$. By construction, the wavelets satisfy the de Rahm co-chain complex, the Hodge decomposition, and that the $k$-dimensional integral of an $r$-form is an $(r-k)$-form. They also verify Stokes' theorem for differential forms, with the most efficient finite dimensional approximation attained using directionally localized, curvelet- or ridgelet-like forms. The construction of $Ψ\mathrm{ec}$ builds on the geometric simplicity of the exterior calculus in the Fourier domain. We establish this structure by extending existing results on the Fourier transform of differential forms to a frequency description of the exterior calculus, including, for example, a Plancherel theorem for forms and a description of the symbols of all important operators.
△ Less
Submitted 5 October, 2020; v1 submitted 29 November, 2018;
originally announced November 2018.
-
A Local Fourier Slice Theorem
Authors:
Christian Lessig
Abstract:
We present a local Fourier slice equation that enables local and sparse projection of a signal. Our result exploits that a slice in frequency space is an iso-parameter set in spherical coordinates. Therefore, the projection of suitable wavelets defined separably in these coordinates can be computed analytically, yielding a sequence of wavelets closed under projection. Our local Fourier slice equat…
▽ More
We present a local Fourier slice equation that enables local and sparse projection of a signal. Our result exploits that a slice in frequency space is an iso-parameter set in spherical coordinates. Therefore, the projection of suitable wavelets defined separably in these coordinates can be computed analytically, yielding a sequence of wavelets closed under projection. Our local Fourier slice equation then realizes projection as reconstruction with "sliced" wavelets with computational costs that scale linearly in the complexity of the projected signal. We numerically evaluate the performance of our local Fourier slice equation for synthetic test data and tomographic reconstruction, demonstrating that locality and sparsity can significantly reduce computation times and memory requirements.
△ Less
Submitted 7 September, 2018; v1 submitted 25 June, 2018;
originally announced June 2018.
-
Divergence Free Polar Wavelets for the Analysis and Representation of Fluid Flows
Authors:
Christian Lessig
Abstract:
We present a Parseval tight wavelet frame for the representation and analysis of velocity vector fields of incompressible fluids. Our wavelets have closed form expressions in the frequency and spatial domains, are divergence free in the ideal, analytic sense, have a multi-resolution structure and fast transforms, and an intuitive correspondence to common flow phenomena. Our construction also allow…
▽ More
We present a Parseval tight wavelet frame for the representation and analysis of velocity vector fields of incompressible fluids. Our wavelets have closed form expressions in the frequency and spatial domains, are divergence free in the ideal, analytic sense, have a multi-resolution structure and fast transforms, and an intuitive correspondence to common flow phenomena. Our construction also allows for well defined directional selectivity, e.g. to model the behavior of divergence free vector fields in the vicinity of boundaries or to represent highly directional features like in a von Kármán vortex street. We demonstrate the practicality and efficiency of our construction by analyzing the representation of different divergence free vector fields in our wavelets.
△ Less
Submitted 30 September, 2018; v1 submitted 5 May, 2018;
originally announced May 2018.
-
Polar Wavelets in Space
Authors:
Christian Lessig
Abstract:
Recent work introduced a unified framework for steerable and directional wavelets in two and three dimensions that ensures many desirable properties, such as a multi-scale structure, fast transforms, and a flexible angular localization. We show that, for an appropriate choice for the radial window function, these wavelets also have closed form expressions for, among other things, the spatial repre…
▽ More
Recent work introduced a unified framework for steerable and directional wavelets in two and three dimensions that ensures many desirable properties, such as a multi-scale structure, fast transforms, and a flexible angular localization. We show that, for an appropriate choice for the radial window function, these wavelets also have closed form expressions for, among other things, the spatial representation, the filter taps for the fast transform, and the frame representation of the Laplace operator. The numerical practicality and benefits of our work are demonstrated using signal estimation from non-uniform, point-wise samples, as required for example in ray tracing, and for reconstructing a signal over a lower-dimensional sub-manifold, with applications for instance in medical imaging.
△ Less
Submitted 5 May, 2018;
originally announced May 2018.
-
Bendlets: A Second-Order Shearlet Transform with Bent Elements
Authors:
Christian Lessig,
Philipp Petersen,
Martin Schäfer
Abstract:
We introduce bendlets, a shearlet-like system that is based on anisotropic scaling, translation, shearing, and bending of a compactly supported generator. With shearing being linear and bending quadratic in spatial coordinates, bendlets provide what we term a second-order shearlet system. As we show in this article, the decay rates of the associated transform enable the precise characterization of…
▽ More
We introduce bendlets, a shearlet-like system that is based on anisotropic scaling, translation, shearing, and bending of a compactly supported generator. With shearing being linear and bending quadratic in spatial coordinates, bendlets provide what we term a second-order shearlet system. As we show in this article, the decay rates of the associated transform enable the precise characterization of location, orientation and curvature of discontinuities in piecewise constant images. These results yield an improvement over existing directional representation systems where curvature only controls the constant of the decay rate of the transform. We also detail the construction of shearlet systems of arbitrary order. A practical implementation of bendlets is provided as an extension of the ShearLab toolbox, which we use to verify our theoretical classification results.
△ Less
Submitted 15 May, 2017; v1 submitted 19 July, 2016;
originally announced July 2016.
