-
Leveraging HPC Profiling & Tracing Tools to Understand the Performance of Particle-in-Cell Monte Carlo Simulations
Authors:
Jeremy J. Williams,
David Tskhakaya,
Stefan Costea,
Ivy B. Peng,
Marta Garcia-Gasulla,
Stefano Markidis
Abstract:
Large-scale plasma simulations are critical for designing and develo** next-generation fusion energy devices and modeling industrial plasmas. BIT1 is a massively parallel Particle-in-Cell code designed for specifically studying plasma material interaction in fusion devices. Its most salient characteristic is the inclusion of collision Monte Carlo models for different plasma species. In this work…
▽ More
Large-scale plasma simulations are critical for designing and develo** next-generation fusion energy devices and modeling industrial plasmas. BIT1 is a massively parallel Particle-in-Cell code designed for specifically studying plasma material interaction in fusion devices. Its most salient characteristic is the inclusion of collision Monte Carlo models for different plasma species. In this work, we characterize single node, multiple nodes, and I/O performances of the BIT1 code in two realistic cases by using several HPC profilers, such as perf, IPM, Extrae/Paraver, and Darshan tools. We find that the BIT1 sorting function on-node performance is the main performance bottleneck. Strong scaling tests show a parallel performance of 77% and 96% on 2,560 MPI ranks for the two test cases. We demonstrate that communication, load imbalance and self-synchronization are important factors impacting the performance of the BIT1 on large-scale runs.
△ Less
Submitted 28 June, 2023;
originally announced June 2023.
-
Lessons learned from a performance analysis and optimization of a multiscale cellular simulation
Authors:
Marc Clascà,
Marta Garcia-Gasulla,
Arnau Montagud,
Jose Carbonell Caballero,
Alfonso Valencia
Abstract:
This work presents a comprehensive performance analysis and optimization of a multiscale agent-based cellular simulation. The optimizations applied are guided by detailed performance analysis and include memory management, load balance, and a locality-aware parallelization. The outcome of this paper is not only the speedup of 2.4x achieved by the optimized version with respect to the original Phys…
▽ More
This work presents a comprehensive performance analysis and optimization of a multiscale agent-based cellular simulation. The optimizations applied are guided by detailed performance analysis and include memory management, load balance, and a locality-aware parallelization. The outcome of this paper is not only the speedup of 2.4x achieved by the optimized version with respect to the original PhysiCell code, but also the lessons learned and best practices when develo** parallel HPC codes to obtain efficient and highly performant applications, especially in the computational biology field.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
Software Development Vehicles to enable extended and early co-design: a RISC-V and HPC case of study
Authors:
Filippo Mantovani,
Pablo Vizcaino,
Fabio Banchelli,
Marta Garcia-Gasulla,
Roger Ferrer,
Giorgos Ieronymakis,
Nikos Dimou,
Vassilis Papaefstathiou,
Jesus Labarta
Abstract:
Prototy** HPC systems with low-to-mid technology readiness level (TRL) systems is critical for providing feedback to hardware designers, the system software team (e.g., compiler developers), and early adopters from the scientific community. The typical approach to hardware design and HPC system prototy** often limits feedback or only allows it at a late stage. In this paper, we present a set o…
▽ More
Prototy** HPC systems with low-to-mid technology readiness level (TRL) systems is critical for providing feedback to hardware designers, the system software team (e.g., compiler developers), and early adopters from the scientific community. The typical approach to hardware design and HPC system prototy** often limits feedback or only allows it at a late stage. In this paper, we present a set of tools for co-designing HPC systems, called software development vehicles (SDV). We use an innovative RISC-V design as a demonstrator, which includes a scalar CPU and a vector processing unit capable of operating large vectors up to 16 kbits. We provide an incremental methodology and early tangible evidence of the co-design process that provide feedback to improve both architecture and system software at a very early stage of system development.
△ Less
Submitted 1 June, 2023;
originally announced June 2023.
-
Runtime-Adaptable Selective Performance Instrumentation
Authors:
Sebastian Kreutzer,
Christian Iwainsky,
Marta Garcia-Gasulla,
Victor Lopez,
Christian Bischof
Abstract:
Automated code instrumentation, i.e. the insertion of measurement hooks into a target application by the compiler, is an established technique for collecting reliable, fine-grained performance data. The set of functions to instrument has to be selected with care, as instrumenting every available function typically yields too large a runtime overhead, thus skewing the measurement. No "one-suits-all…
▽ More
Automated code instrumentation, i.e. the insertion of measurement hooks into a target application by the compiler, is an established technique for collecting reliable, fine-grained performance data. The set of functions to instrument has to be selected with care, as instrumenting every available function typically yields too large a runtime overhead, thus skewing the measurement. No "one-suits-all" selection mechanism exists, since the instrumentation decision is dependent on the measurement objective, the limit for tolerable runtime overhead and peculiarities of the target application. The Compiler-assisted Performance Instrumentation (CaPI) tool assists in creating such instrumentation configurations, by enabling the user to combine different selection mechanisms as part of a configurable selection pipeline, operating on a statically constructed whole-program call-graph. Previously, CaPI relied on a static instrumentation workflow which made the process of refining the initial selection quite cumbersome for large-scale codes, as the application had to be recompiled after each adjustment. In this work, we present new runtime-adaptable instrumentation capabilities for CaPI which do not require recompilation when instrumentation changes are made. To this end, the XRay instrumentation feature of the LLVM compiler was extended to support the instrumentation of shared dynamic objects. An XRay-compatible runtime system was added to CaPI that instruments selected functions at program start, thereby significantly reducing the required time for selection refinements. Furthermore, an interface to the TALP tool for recording parallel efficiency metrics was implemented, alongside a specialized selection module for creating suitable coarse-grained region instrumentations.
△ Less
Submitted 20 March, 2023;
originally announced March 2023.
-
A portable coding strategy to exploit vectorization on combustion simulations
Authors:
Fabio Banchelli,
Guillermo Oyarzun,
Marta Garcia-Gasulla,
Filippo Mantovani,
Ambrus Both,
Guillaume Houzeaux,
Daniel Mira
Abstract:
The complexity of combustion simulations demands the latest high-performance computing tools to accelerate its time-to-solution results. A current trend on HPC systems is the utilization of CPUs with SIMD or vector extensions to exploit data parallelism. Our work proposes a strategy to improve the automatic vectorization of finite element-based scientific codes. The approach applies a parametric c…
▽ More
The complexity of combustion simulations demands the latest high-performance computing tools to accelerate its time-to-solution results. A current trend on HPC systems is the utilization of CPUs with SIMD or vector extensions to exploit data parallelism. Our work proposes a strategy to improve the automatic vectorization of finite element-based scientific codes. The approach applies a parametric configuration to the data structures to help the compiler detect the block of codes that can take advantage of vector computation while maintaining the code portable. A detailed analysis of the computational impact of this methodology on the different stages of a CFD solver is studied on the PRECCINSTA burner simulation. Our parametric implementation has proven to help the compiler generate more vector instructions in the assembly operation: this results in a reduction of up to 9.3 times of the total executed instruction maintaining constant the Instructions Per Cycle and the CPU frequency. The proposed strategy improves the performance of the CFD case under study up to 4.67 times on the MareNostrum 4 supercomputer.
△ Less
Submitted 21 October, 2022;
originally announced October 2022.
-
Dynamic resource allocation for efficient parallel CFD simulations
Authors:
G. Houzeaux,
R. M. Badia,
R. Borrell,
D. Dosimont,
J. Ejarque,
M. Garcia-Gasulla,
V. López
Abstract:
CFD users of supercomputers usually resort to rule-of-thumb methods to select the number of subdomains (partitions) when relying on MPI-based parallelization. One common approach is to set a minimum number of elements or cells per subdomain, under which the parallel efficiency of the code is "known" to fall below a subjective level, say 80%. The situation is even worse when the user is not aware o…
▽ More
CFD users of supercomputers usually resort to rule-of-thumb methods to select the number of subdomains (partitions) when relying on MPI-based parallelization. One common approach is to set a minimum number of elements or cells per subdomain, under which the parallel efficiency of the code is "known" to fall below a subjective level, say 80%. The situation is even worse when the user is not aware of the "good" practices for the given code and a huge amount of resources can thus be wasted. This work presents an elastic computing methodology to adapt at runtime the resources allocated to a simulation automatically. The criterion to control the required resources is based on a runtime measure of the communication efficiency of the execution. According to some analytical estimates, the resources are then expanded or reduced to fulfil this criterion and eventually execute an efficient simulation.
△ Less
Submitted 29 June, 2022; v1 submitted 17 December, 2021;
originally announced December 2021.
-
Performance and energy consumption of HPC workloads on a cluster based on Arm ThunderX2 CPU
Authors:
Filippo Mantovani,
Marta Garcia-Gasulla,
José Gracia,
Esteban Stafford,
Fabio Banchelli,
Marc Josep-Fabrego,
Joel Criado-Ledesma,
Mathias Nachtmann
Abstract:
In this paper, we analyze the performance and energy consumption of an Arm-based high-performance computing (HPC) system developed within the European project Mont-Blanc 3. This system, called Dibona, has been integrated by ATOS/Bull, and it is powered by the latest Marvell's CPU, ThunderX2. This CPU is the same one that powers the Astra supercomputer, the first Arm-based supercomputer entering th…
▽ More
In this paper, we analyze the performance and energy consumption of an Arm-based high-performance computing (HPC) system developed within the European project Mont-Blanc 3. This system, called Dibona, has been integrated by ATOS/Bull, and it is powered by the latest Marvell's CPU, ThunderX2. This CPU is the same one that powers the Astra supercomputer, the first Arm-based supercomputer entering the Top500 in November 2018. We study from micro-benchmarks up to large production codes. We include an interdisciplinary evaluation of three scientific applications (a finite-element fluid dynamics code, a smoothed particle hydrodynamics code, and a lattice Boltzmann code) and the Graph 500 benchmark, focusing on parallel and energy efficiency as well as studying their scalability up to thousands of Armv8 cores. For comparison, we run the same tests on state-of-the-art x86 nodes included in Dibona and the Tier-0 supercomputer MareNostrum4. Our experiments show that the ThunderX2 has a 25% lower performance on average, mainly due to its small vector unit yet somewhat compensated by its 30% wider links between the CPU and the main memory. We found that the software ecosystem of the Armv8 architecture is comparable to the one available for Intel. Our results also show that ThunderX2 delivers similar or better energy-to-solution and scalability, proving that Arm-based chips are legitimate contenders in the market of next-generation HPC systems.
△ Less
Submitted 10 July, 2020; v1 submitted 9 July, 2020;
originally announced July 2020.
-
Heterogeneous CPU/GPU co-execution of CFD simulations on the POWER9 architecture: Application to airplane aerodynamics
Authors:
R. Borrell,
D. Dosimont,
M. Garcia-Gasulla,
G. Houzeaux,
O. Lehmkuhl,
V. Mehta,
H. Owen,
M. Vazquez,
G. Oyarzun
Abstract:
High fidelity Computational Fluid Dynamics simulations are generally associated with large computing requirements, which are progressively acute with each new generation of supercomputers. However, significant research efforts are required to unlock the computing power of leading-edge systems, currently referred to as pre-Exascale systems, based on increasingly complex architectures. In this paper…
▽ More
High fidelity Computational Fluid Dynamics simulations are generally associated with large computing requirements, which are progressively acute with each new generation of supercomputers. However, significant research efforts are required to unlock the computing power of leading-edge systems, currently referred to as pre-Exascale systems, based on increasingly complex architectures. In this paper, we present the approach implemented in the computational mechanics code Alya. We describe in detail the parallelization strategy implemented to fully exploit the different levels of parallelism, together with a novel co-execution method for the efficient utilization of heterogeneous CPU/GPU architectures. The latter is based on a multi-code co-execution approach with a dynamic load balancing mechanism. The assessment of the performance of all the proposed strategies has been carried out for airplane simulations on the POWER9 architecture accelerated with NVIDIA Volta V100 GPUs.
△ Less
Submitted 6 July, 2020; v1 submitted 12 May, 2020;
originally announced May 2020.
-
MPI+X: task-based parallelization and dynamic load balance of finite element assembly
Authors:
Marta Garcia-Gasulla,
Guillaume Houzeaux,
Roger Ferrer,
Antoni Artigues,
Victor López,
Jesús Labarta,
Mariano Vázquez
Abstract:
The main computing tasks of a finite element code(FE) for solving partial differential equations (PDE's) are the algebraic system assembly and the iterative solver. This work focuses on the first task, in the context of a hybrid MPI+X paradigm. Although we will describe algorithms in the FE context, a similar strategy can be straightforwardly applied to other discretization methods, like the finit…
▽ More
The main computing tasks of a finite element code(FE) for solving partial differential equations (PDE's) are the algebraic system assembly and the iterative solver. This work focuses on the first task, in the context of a hybrid MPI+X paradigm. Although we will describe algorithms in the FE context, a similar strategy can be straightforwardly applied to other discretization methods, like the finite volume method. The matrix assembly consists of a loop over the elements of the MPI partition to compute element matrices and right-hand sides and their assemblies in the local system to each MPI partition. In a MPI+X hybrid parallelism context, X has consisted traditionally of loop parallelism using OpenMP. Several strategies have been proposed in the literature to implement this loop parallelism, like coloring or substructuring techniques to circumvent the race condition that appears when assembling the element system into the local system. The main drawback of the first technique is the decrease of the IPC due to bad spatial locality. The second technique avoids this issue but requires extensive changes in the implementation, which can be cumbersome when several element loops should be treated. We propose an alternative, based on the task parallelism of the element loop using some extensions to the OpenMP programming model. The taskification of the assembly solves both aforementioned problems. In addition, dynamic load balance will be applied using the DLB library, especially efficient in the presence of hybrid meshes, where the relative costs of the different elements is impossible to estimate a priori. This paper presents the proposed methodology, its implementation and its validation through the solution of large computational mechanics problems up to 16k cores.
△ Less
Submitted 9 May, 2018;
originally announced May 2018.
