-
A highly scalable approach to solving linear systems using two-stage multisplitting
Authors:
Nick Brown,
J. Mark Bull,
Iain Bethune
Abstract:
Iterative methods for solving large sparse systems of linear equations are widely used in many HPC applications. Extreme scaling of these methods can be difficult, however, since global communication to form dot products is typically required at every iteration.
To try to overcome this limitation we propose a hybrid approach, where the matrix is partitioned into blocks. Within each block, we use…
▽ More
Iterative methods for solving large sparse systems of linear equations are widely used in many HPC applications. Extreme scaling of these methods can be difficult, however, since global communication to form dot products is typically required at every iteration.
To try to overcome this limitation we propose a hybrid approach, where the matrix is partitioned into blocks. Within each block, we use a highly optimised (parallel) conventional solver, but we then couple the blocks together using block Jacobi or some other multisplitting technique that can be implemented in either a synchronous or an asynchronous fashion. This allows us to limit the block size to the point where the conventional iterative methods no longer scale, and to avoid global communication (and possibly synchronisation) across all processes.
Our block framework has been built to use PETSc, a popular scientific suite for solving sparse linear systems, as the synchronous intra-block solver, and we demonstrate results on up to 32768 cores of a Cray XE6 system. At this scale, the conventional solvers are still more efficient, though trends suggest that the hybrid approach may be beneficial at higher core counts.
△ Less
Submitted 26 September, 2020;
originally announced September 2020.
-
CP2K: An Electronic Structure and Molecular Dynamics Software Package -- Quickstep: Efficient and Accurate Electronic Structure Calculations
Authors:
Thomas D. Kühne,
Marcella Iannuzzi,
Mauro Del Ben,
Vladimir V. Rybkin,
Patrick Seewald,
Frederick Stein,
Teodoro Laino,
Rustam Z. Khaliullin,
Ole Schütt,
Florian Schiffmann,
Dorothea Golze,
Jan Wilhelm,
Sergey Chulkov,
Mohammad Hossein Bani-Hashemian,
Valéry Weber,
Urban Borstnik,
Mathieu Taillefumier,
Alice Shoshana Jakobovits,
Alfio Lazzaro,
Hans Pabst,
Tiziano Müller,
Robert Schade,
Manuel Guidon,
Samuel Andermatt,
Nico Holmberg
, et al. (14 additional authors not shown)
Abstract:
CP2K is an open source electronic structure and molecular dynamics software package to perform atomistic simulations of solid-state, liquid, molecular and biological systems. It is especially aimed at massively-parallel and linear-scaling electronic structure methods and state-of-the-art ab-initio molecular dynamics simulations. Excellent performance for electronic structure calculations is achiev…
▽ More
CP2K is an open source electronic structure and molecular dynamics software package to perform atomistic simulations of solid-state, liquid, molecular and biological systems. It is especially aimed at massively-parallel and linear-scaling electronic structure methods and state-of-the-art ab-initio molecular dynamics simulations. Excellent performance for electronic structure calculations is achieved using novel algorithms implemented for modern high-performance computing systems. This review revisits the main capabilities of CP2k to perform efficient and accurate electronic structure simulations. The emphasis is put on density functional theory and multiple post-Hartree-Fock methods using the Gaussian and plane wave approach and its augmented all-electron extension.
△ Less
Submitted 11 March, 2020; v1 submitted 8 March, 2020;
originally announced March 2020.
-
MIST: A Simple and Efficient Molecular Dynamics Abstraction Library for Integrator Development
Authors:
Iain Bethune,
Ralf Banisch,
Elena Breitmoser,
Antonia B K Collis,
Gordon Gibb,
Gianpaolo Gobbo,
Charles Matthews,
Graeme J Ackland,
Benedict J Leimkuhler
Abstract:
We present MIST, the Molecular Integration Simulation Toolkit, a lightweight and efficient software library written in C++ which provides an abstract in- terface to common molecular dynamics codes, enabling rapid and portable development of new integration schemes for molecular dynamics. The initial release provides plug-in interfaces to NAMD-Lite, GROMACS and Amber, and includes several standard…
▽ More
We present MIST, the Molecular Integration Simulation Toolkit, a lightweight and efficient software library written in C++ which provides an abstract in- terface to common molecular dynamics codes, enabling rapid and portable development of new integration schemes for molecular dynamics. The initial release provides plug-in interfaces to NAMD-Lite, GROMACS and Amber, and includes several standard integration schemes, a constraint solver, tem- perature control using Langevin Dynamics, and two tempering schemes. We describe the architecture and functionality of the library and the C and For- tran APIs which can be used to interface additional MD codes to MIST. We show, for a range of test systems, that MIST introduces negligible overheads for serial, shared-memory parallel, and GPU-accelerated cases, except for Amber where the native integrators run directly on the GPU itself. As a demonstration of the capabilities of MIST, we describe a simulated tempering simulation used to study the free energy landscape of Alanine-12 in both vacuum and detailed solvent conditions.
△ Less
Submitted 8 October, 2018; v1 submitted 18 May, 2018;
originally announced May 2018.
-
Porting of the DBCSR library for Sparse Matrix-Matrix Multiplications to Intel Xeon Phi systems
Authors:
Iain Bethune,
Andeas Gloess,
Juerg Hutter,
Alfio Lazzaro,
Hans Pabst,
Fiona Reid
Abstract:
Multiplication of two sparse matrices is a key operation in the simulation of the electronic structure of systems containing thousands of atoms and electrons. The highly optimized sparse linear algebra library DBCSR (Distributed Block Compressed Sparse Row) has been specifically designed to efficiently perform such sparse matrix-matrix multiplications. This library is the basic building block for…
▽ More
Multiplication of two sparse matrices is a key operation in the simulation of the electronic structure of systems containing thousands of atoms and electrons. The highly optimized sparse linear algebra library DBCSR (Distributed Block Compressed Sparse Row) has been specifically designed to efficiently perform such sparse matrix-matrix multiplications. This library is the basic building block for linear scaling electronic structure theory and low scaling correlated methods in CP2K. It is parallelized using MPI and OpenMP, and can exploit GPU accelerators by means of CUDA. We describe a performance comparison of DBCSR on systems with Intel Xeon Phi Knights Landing (KNL) processors, with respect to systems with Intel Xeon CPUs (including systems with GPUs). We find that the DBCSR on Cray XC40 KNL-based systems is 11%-14% slower than on a hybrid Cray XC50 with Nvidia P100 cards, at the same number of nodes. When compared to a Cray XC40 system equipped with dual-socket Intel Xeon CPUs, the KNL is up to 24% faster.
△ Less
Submitted 21 August, 2017; v1 submitted 11 August, 2017;
originally announced August 2017.
-
ExTASY: Scalable and Flexible Coupling of MD Simulations and Advanced Sampling Techniques
Authors:
Vivekanandan Balasubramanian,
Iain Bethune,
Ardita Shkurti,
Elena Breitmoser,
Eugen Hruska,
Cecilia Clementi,
Charles Laughton,
Shantenu Jha
Abstract:
For many macromolecular systems the accurate sampling of the relevant regions on the potential energy surface cannot be obtained by a single, long Molecular Dynamics (MD) trajectory. New approaches are required to promote more efficient sampling. We present the design and implementation of the Extensible Toolkit for Advanced Sampling and analYsis (ExTASY) for building and executing advanced sampli…
▽ More
For many macromolecular systems the accurate sampling of the relevant regions on the potential energy surface cannot be obtained by a single, long Molecular Dynamics (MD) trajectory. New approaches are required to promote more efficient sampling. We present the design and implementation of the Extensible Toolkit for Advanced Sampling and analYsis (ExTASY) for building and executing advanced sampling workflows on HPC systems. ExTASY provides Python based "templated scripts" that interface to an interoperable and high-performance pilot-based run time system, which abstracts the complexity of managing multiple simulations. ExTASY supports the use of existing highly-optimised parallel MD code and their coupling to analysis tools based upon collective coordinates which do not require a priori knowledge of the system to bias. We describe two workflows which both couple large "ensembles" of relatively short MD simulations with analysis tools to automatically analyse the generated trajectories and identify molecular conformational structures that will be used on-the-fly as new starting points for further "simulation-analysis" iterations. One of the workflows leverages the Locally Scaled Diffusion Maps technique; the other makes use of Complementary Coordinates techniques to enhance sampling and generate start-points for the next generation of MD simulations. We show that the ExTASY tools have been deployed on a range of HPC systems including ARCHER (Cray CX30), Blue Waters (Cray XE6/XK7), and Stampede (Linux cluster), and that good strong scaling can be obtained up to 1000s of MD simulations, independent of the size of each simulation. We discuss how ExTASY can be easily extended or modified by end-users to build their own workflows, and ongoing work to improve the usability and robustness of ExTASY.
△ Less
Submitted 31 May, 2016;
originally announced June 2016.
-
Simplified TPLS as a learning tool for high-performance computational fluid dynamics
Authors:
James Fannon,
Jean-Christophe Loiseau,
Prashant Valluri,
Iain Bethune,
Lennon O'Naraigh
Abstract:
We introduce a modified and simplified version of the pre-existing fully parallelized three-dimensional Navier--Stokes flow solver known as TPLS. We demonstrate how the simplified version can be used as a pedagogical tool for the study of computational fluid dynamics and parallel computing. TPLS is at its heart a two-phase flow solver, and uses calls to a range of external libraries to accelerate…
▽ More
We introduce a modified and simplified version of the pre-existing fully parallelized three-dimensional Navier--Stokes flow solver known as TPLS. We demonstrate how the simplified version can be used as a pedagogical tool for the study of computational fluid dynamics and parallel computing. TPLS is at its heart a two-phase flow solver, and uses calls to a range of external libraries to accelerate its performance. However, in the present context we narrow the focus of the study to basic hydrodynamics and parallel computing techniques, and the code is therefore simplified and modified to simulate pressure-driven single-phase flow in a channel, using only relatively simple Fortran 90 code with MPI parallelization, but no calls to any other external libraries. The modified code is analysed in order to both validate its accuracy and investigate its scalability up to 1000 CPU cores. Simulations are performed for several benchmark cases in pressure-driven channel flow, including a turbulent simulation, wherein the turbulence is incorporated via the large-eddy simulation technique. The work may be of use to advanced undergraduate and graduate students as an introductory study in computational fluid dynamics, while also providing insight for those interested in more general aspects of high-performance computing.
△ Less
Submitted 24 November, 2015;
originally announced November 2015.
