-
High-performance finite elements with MFEM
Authors:
Julian Andrej,
Nabil Atallah,
Jan-Phillip Bäcker,
John Camier,
Dylan Copeland,
Veselin Dobrev,
Yohann Dudouit,
Tobias Duswald,
Brendan Keith,
Dohyun Kim,
Tzanio Kolev,
Boyan Lazarov,
Ketan Mittal,
Will Pazner,
Socratis Petrides,
Syun'ichi Shiraiwa,
Mark Stowell,
Vladimir Tomov
Abstract:
The MFEM (Modular Finite Element Methods) library is a high-performance C++ library for finite element discretizations. MFEM supports numerous types of finite element methods and is the discretization engine powering many computational physics and engineering applications across a number of domains. This paper describes some of the recent research and development in MFEM, focusing on performance p…
▽ More
The MFEM (Modular Finite Element Methods) library is a high-performance C++ library for finite element discretizations. MFEM supports numerous types of finite element methods and is the discretization engine powering many computational physics and engineering applications across a number of domains. This paper describes some of the recent research and development in MFEM, focusing on performance portability across leadership-class supercomputing facilities, including exascale supercomputers, as well as new capabilities and functionality, enabling a wider range of applications. Much of this work was undertaken as part of the Department of Energy's Exascale Computing Project (ECP) in collaboration with the Center for Efficient Exascale Discretizations (CEED).
△ Less
Submitted 24 February, 2024;
originally announced February 2024.
-
Mixed-Order Meshes through rp-adaptivity for Surface Fitting to Implicit Geometries
Authors:
Ketan Mittal,
Veselin A. Dobrev,
Patrick Knupp,
Tzanio Kolev,
Franck Ledoux,
Claire Roche,
Vladimir Z. Tomov
Abstract:
Computational analysis with the finite element method requires geometrically accurate meshes. It is well known that high-order meshes can accurately capture curved surfaces with fewer degrees of freedom in comparison to low-order meshes. Existing techniques for high-order mesh generation typically output meshes with same polynomial order for all elements. However, high order elements away from cur…
▽ More
Computational analysis with the finite element method requires geometrically accurate meshes. It is well known that high-order meshes can accurately capture curved surfaces with fewer degrees of freedom in comparison to low-order meshes. Existing techniques for high-order mesh generation typically output meshes with same polynomial order for all elements. However, high order elements away from curvilinear boundaries or interfaces increase the computational cost of the simulation without increasing geometric accuracy. In prior work, we have presented one such approach for generating body-fitted uniform-order meshes that takes a given mesh and morphs it to align with the surface of interest prescribed as the zero isocontour of a level-set function. We extend this method to generate mixed-order meshes such that curved surfaces of the domain are discretized with high-order elements, while low-order elements are used elsewhere. Numerical experiments demonstrate the robustness of the approach and show that it can be used to generate mixed-order meshes that are much more efficient than high uniform-order meshes. The proposed approach is purely algebraic, and extends to different types of elements (quadrilaterals/triangles/tetrahedron/hexahedra) in two- and three-dimensions.
△ Less
Submitted 29 January, 2024;
originally announced January 2024.
-
High-Order Mesh Morphing for Boundary and Interface Fitting to Implicit Geometries
Authors:
Jorge-Luis Barrera,
Tzanio Kolev,
Ketan Mittal,
Vladimir Tomov
Abstract:
We propose a method that morphs high-orger meshes such that their boundaries and interfaces coincide/align with implicitly defined geometries. Our focus is particularly on the case when the target surface is prescribed as the zero isocontour of a smooth discrete function. Common examples of this scenario include using level set functions to represent material interfaces in multimaterial configurat…
▽ More
We propose a method that morphs high-orger meshes such that their boundaries and interfaces coincide/align with implicitly defined geometries. Our focus is particularly on the case when the target surface is prescribed as the zero isocontour of a smooth discrete function. Common examples of this scenario include using level set functions to represent material interfaces in multimaterial configurations, and evolving geometries in shape and topology optimization. The proposed method formulates the mesh optimization problem as a variational minimization of the sum of a chosen mesh-quality metric using the Target-Matrix Optimization Paradigm (TMOP) and a penalty term that weakly forces the selected faces of the mesh to align with the target surface. The distinct features of the method are use of a source mesh to represent the level set function with sufficient accuracy, and adaptive strategies for setting the penalization weight and selecting the faces of the mesh to be fit to the target isocontour of the level set field. We demonstrate that the proposed method is robust for generating boundary- and interface-fitted meshes for curvilinear domains using different element types in 2D and 3D.
△ Less
Submitted 6 February, 2023; v1 submitted 9 August, 2022;
originally announced August 2022.
-
Accelerating High-Order Mesh Optimization Using Finite Element Partial Assembly on GPUs
Authors:
Jean-Sylvain Camier,
Veselin Dobrev,
Patrick Knupp,
Tzanio Kolev,
Ketan Mittal,
Robert Rieben,
Vladimir Tomov
Abstract:
In this paper we present a new GPU-oriented mesh optimization method based on high-order finite elements. Our approach relies on node movement with fixed topology, through the Target-Matrix Optimization Paradigm (TMOP) and uses a global nonlinear solve over the whole computational mesh, i.e., all mesh nodes are moved together. A key property of the method is that the mesh optimization process is r…
▽ More
In this paper we present a new GPU-oriented mesh optimization method based on high-order finite elements. Our approach relies on node movement with fixed topology, through the Target-Matrix Optimization Paradigm (TMOP) and uses a global nonlinear solve over the whole computational mesh, i.e., all mesh nodes are moved together. A key property of the method is that the mesh optimization process is recast in terms of finite element operations, which allows us to utilize recent advances in the field of GPU-accelerated high-order finite element algorithms. For example, we reduce data motion by using tensor factorization and matrix-free methods, which have superior performance characteristics compared to traditional full finite element matrix assembly and offer advantages for GPU-based HPC hardware. We describe the major mathematical components of the method along with their efficient GPU-oriented implementation. In addition, we propose an easily reproducible mesh optimization test that can serve as a performance benchmark for the mesh optimization community.
△ Less
Submitted 1 December, 2022; v1 submitted 23 May, 2022;
originally announced May 2022.
-
Matrix-free approaches for GPU acceleration of a high-order finite element hydrodynamics application using MFEM, Umpire, and RAJA
Authors:
Arturo Vargas,
Thomas M. Stitt,
Kenneth Weiss,
Vladimir Z. Tomov,
Jean-Sylvain Camier,
Tzanio Kolev,
Robert N. Rieben
Abstract:
With the introduction of advanced heterogeneous computing architectures based on GPU accelerators, large-scale production codes have had to rethink their numerical algorithms and incorporate new programming models and memory management strategies in order to run efficiently on the latest supercomputers. In this work we discuss our co-design strategy to address these challenges and achieve performa…
▽ More
With the introduction of advanced heterogeneous computing architectures based on GPU accelerators, large-scale production codes have had to rethink their numerical algorithms and incorporate new programming models and memory management strategies in order to run efficiently on the latest supercomputers. In this work we discuss our co-design strategy to address these challenges and achieve performance and portability with MARBL, a next-generation multi-physics code in development at Lawrence Livermore National Laboratory. We present a two-fold approach, wherein new hardware is used to motivate both new algorithms and new abstraction layers, resulting in a single source application code suitable for a variety of platforms. Focusing on MARBL's ALE hydrodynamics package, we demonstrate scalability on different platforms and highlight that many of our innovations have been contributed back to open-source software libraries, such as MFEM (finite element algorithms) and RAJA (kernel abstractions).
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
GPU Algorithms for Efficient Exascale Discretizations
Authors:
Ahmad Abdelfattah,
Valeria Barra,
Natalie Beams,
Ryan Bleile,
Jed Brown,
Jean-Sylvain Camier,
Robert Carson,
Noel Chalmers,
Veselin Dobrev,
Yohann Dudouit,
Paul Fischer,
Ali Karakus,
Stefan Kerkemeier,
Tzanio Kolev,
Yu-Hsiang Lan,
Elia Merzari,
Misun Min,
Malachi Phillips,
Thilina Rathnayake,
Robert Rieben,
Thomas Stitt,
Ananias Tomboulides,
Stanimire Tomov,
Vladimir Tomov,
Arturo Vargas
, et al. (2 additional authors not shown)
Abstract:
In this paper we describe the research and development activities in the Center for Efficient Exascale Discretization within the US Exascale Computing Project, targeting state-of-the-art high-order finite-element algorithms for high-order applications on GPU-accelerated platforms. We discuss the GPU developments in several components of the CEED software stack, including the libCEED, MAGMA, MFEM,…
▽ More
In this paper we describe the research and development activities in the Center for Efficient Exascale Discretization within the US Exascale Computing Project, targeting state-of-the-art high-order finite-element algorithms for high-order applications on GPU-accelerated platforms. We discuss the GPU developments in several components of the CEED software stack, including the libCEED, MAGMA, MFEM, libParanumal, and Nek projects. We report performance and capability improvements in several CEED-enabled applications on both NVIDIA and AMD GPU systems.
△ Less
Submitted 10 September, 2021;
originally announced September 2021.
-
Efficient Exascale Discretizations: High-Order Finite Element Methods
Authors:
Tzanio Kolev,
Paul Fischer,
Misun Min,
Jack Dongarra,
Jed Brown,
Veselin Dobrev,
Tim Warburton,
Stanimire Tomov,
Mark S. Shephard,
Ahmad Abdelfattah,
Valeria Barra,
Natalie Beams,
Jean-Sylvain Camier,
Noel Chalmers,
Yohann Dudouit,
Ali Karakus,
Ian Karlin,
Stefan Kerkemeier,
Yu-Hsiang Lan,
David Medina,
Elia Merzari,
Aleksandr Obabko,
Will Pazner,
Thilina Rathnayake,
Cameron W. Smith
, et al. (5 additional authors not shown)
Abstract:
Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on u…
▽ More
Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on unstructured grids is to use matrix-free/partially-assembled high-order finite element methods, since these methods can increase the accuracy and/or lower the computational time due to reduced data motion. In this paper we provide an overview of the research and development activities in the Center for Efficient Exascale Discretizations (CEED), a co-design center in the Exascale Computing Project that is focused on the development of next-generation discretization software and algorithms to enable a wide range of finite element applications to run efficiently on future hardware. CEED is a research partnership involving more than 30 computational scientists from two US national labs and five universities, including members of the Nek5000, MFEM, MAGMA and PETSc projects. We discuss the CEED co-design activities based on targeted benchmarks, miniapps and discretization libraries and our work on performance optimizations for large-scale GPU architectures. We also provide a broad overview of research and development activities in areas such as unstructured adaptive mesh refinement algorithms, matrix-free linear solvers, high-order data visualization, and list examples of collaborations with several ECP and external applications.
△ Less
Submitted 10 September, 2021;
originally announced September 2021.
-
Reinforcement Learning for Adaptive Mesh Refinement
Authors:
Jiachen Yang,
Tarik Dzanic,
Brenden Petersen,
Jun Kudo,
Ketan Mittal,
Vladimir Tomov,
Jean-Sylvain Camier,
Tuo Zhao,
Hongyuan Zha,
Tzanio Kolev,
Robert Anderson,
Daniel Faissol
Abstract:
Large-scale finite element simulations of complex physical systems governed by partial differential equations (PDE) crucially depend on adaptive mesh refinement (AMR) to allocate computational budget to regions where higher resolution is required. Existing scalable AMR methods make heuristic refinement decisions based on instantaneous error estimation and thus do not aim for long-term optimality o…
▽ More
Large-scale finite element simulations of complex physical systems governed by partial differential equations (PDE) crucially depend on adaptive mesh refinement (AMR) to allocate computational budget to regions where higher resolution is required. Existing scalable AMR methods make heuristic refinement decisions based on instantaneous error estimation and thus do not aim for long-term optimality over an entire simulation. We propose a novel formulation of AMR as a Markov decision process and apply deep reinforcement learning (RL) to train refinement policies directly from simulation. AMR poses a new problem for RL as both the state dimension and available action set changes at every step, which we solve by proposing new policy architectures with differing generality and inductive bias. The model sizes of these policy architectures are independent of the mesh size and hence can be deployed on larger simulations than those used at train time. We demonstrate in comprehensive experiments on static function estimation and time-dependent equations that RL policies can be trained on problems without using ground truth solutions, are competitive with a widely-used error estimator, and generalize to larger, more complex, and unseen test problems.
△ Less
Submitted 21 February, 2023; v1 submitted 1 March, 2021;
originally announced March 2021.
-
MFEM: a modular finite element methods library
Authors:
Robert Anderson,
Julian Andrej,
Andrew Barker,
Jamie Bramwell,
Jean-Sylvain Camier,
Jakub Cerveny,
Veselin Dobrev,
Yohann Dudouit,
Aaron Fisher,
Tzanio Kolev,
Will Pazner,
Mark Stowell,
Vladimir Tomov,
Johann Dahm,
David Medina,
Stefano Zampini
Abstract:
MFEM is an open-source, lightweight, flexible and scalable C++ library for modular finite element methods that features arbitrary high-order finite element meshes and spaces, support for a wide variety of discretization approaches and emphasis on usability, portability, and high-performance computing efficiency. MFEM's goal is to provide application scientists with access to cutting-edge algorithm…
▽ More
MFEM is an open-source, lightweight, flexible and scalable C++ library for modular finite element methods that features arbitrary high-order finite element meshes and spaces, support for a wide variety of discretization approaches and emphasis on usability, portability, and high-performance computing efficiency. MFEM's goal is to provide application scientists with access to cutting-edge algorithms for high-order finite element meshing, discretizations and linear solvers, while enabling researchers to quickly and easily develop and test new algorithms in very general, fully unstructured, high-order, parallel and GPU-accelerated settings. In this paper we describe the underlying algorithms and finite element abstractions provided by MFEM, discuss the software implementation, and illustrate various applications of the library.
△ Less
Submitted 13 July, 2020; v1 submitted 20 November, 2019;
originally announced November 2019.
