-
Convergence Acceleration of Preconditioned CG Solver Based on Error Vector Sampling for a Sequence of Linear Systems
Authors:
Takeshi Iwashita,
Kota Ikehara,
Takeshi Fukaya,
Takeshi Mifune
Abstract:
In this paper, we focus on solving a sequence of linear systems with an identical (or similar) coefficient matrix. For this type of problems, we investigate the subspace correction and deflation methods, which use an auxiliary matrix (subspace) to accelerate the convergence of the iterative method. In practical simulations, these acceleration methods typically work well when the range of the auxil…
▽ More
In this paper, we focus on solving a sequence of linear systems with an identical (or similar) coefficient matrix. For this type of problems, we investigate the subspace correction and deflation methods, which use an auxiliary matrix (subspace) to accelerate the convergence of the iterative method. In practical simulations, these acceleration methods typically work well when the range of the auxiliary matrix contains eigenspaces corresponding to small eigenvalues of the coefficient matrix. We have developed a new algebraic auxiliary matrix construction method based on error vector sampling, in which eigenvectors with small eigenvalues are efficiently identified in a solution process. The generated auxiliary matrix is used for the convergence acceleration in the following solution step. Numerical tests confirm that both subspace correction and deflation methods with the auxiliary matrix can accelerate the solution process of the iterative solver. Furthermore, we examine the applicability of our technique to the estimation of the condition number of the coefficient matrix. The algorithm of preconditioned conjugate gradient (PCG) method with the condition number estimation is also shown.
△ Less
Submitted 16 March, 2022;
originally announced March 2022.
-
Accelerating the SpMV kernel on standard CPUs by exploiting the partially diagonal structures
Authors:
Takeshi Fukaya,
Koki Ishida,
Akie Miura,
Takeshi Iwashita,
Hiroshi Nakashima
Abstract:
Sparse Matrix Vector multiplication (SpMV) is one of basic building blocks in scientific computing, and acceleration of SpMV has been continuously required. In this research, we aim for accelerating SpMV on recent CPUs for sparse matrices that have a specific sparsity structure, namely a diagonally structured sparsity pattern. We focus a hybrid storage format that combines the DIA and CSR formats,…
▽ More
Sparse Matrix Vector multiplication (SpMV) is one of basic building blocks in scientific computing, and acceleration of SpMV has been continuously required. In this research, we aim for accelerating SpMV on recent CPUs for sparse matrices that have a specific sparsity structure, namely a diagonally structured sparsity pattern. We focus a hybrid storage format that combines the DIA and CSR formats, so-called the HDC format. First, we recall the importance of introducing cache blocking techniques into HDC-based SpMV kernels. Next, based on the observation of the cache blocked kernel, we present a modified version of the HDC formats, which we call the M-HDC format, in which partial diagonal structures are expected to be more efficiently picked up. For these SpMV kernels, we theoretically analyze the expected performance improvement based on performance models. Then, we conduct comprehensive experiments on state-of-the-art multi-core CPUs. By the experiments using typical matrices, we clarify the detailed performance characteristics of each SpMV kernel. We also evaluate the performance for matrices appearing in practical applications and demonstrate that our approach can accelerate SpMV for some of them. Through the present paper, we demonstrate the effectiveness of exploiting partial diagonal structures by the M-HDC format as a promising approach to accelerating SpMV on CPUs for a certain kind of practical sparse matrices.
△ Less
Submitted 11 May, 2021;
originally announced May 2021.
-
An Integer Arithmetic-Based Sparse Linear Solver Using a GMRES Method and Iterative Refinement
Authors:
Takeshi Iwashita,
Kengo Suzuki,
Takeshi Fukaya
Abstract:
In this paper, we develop a (preconditioned) GMRES solver based on integer arithmetic, and introduce an iterative refinement framework for the solver. We describe the data format for the coefficient matrix and vectors for the solver that is based on integer or fixed-point numbers. To avoid overflow in calculations, we introduce initial scaling and logical shifts (adjustments) of operands in arithm…
▽ More
In this paper, we develop a (preconditioned) GMRES solver based on integer arithmetic, and introduce an iterative refinement framework for the solver. We describe the data format for the coefficient matrix and vectors for the solver that is based on integer or fixed-point numbers. To avoid overflow in calculations, we introduce initial scaling and logical shifts (adjustments) of operands in arithmetic operations. We present the approach for operand shifts, considering the characteristics of the GMRES algorithm. Numerical tests demonstrate that the integer arithmetic-based solver with iterative refinement has comparable solver performance in terms of convergence to the standard solver based on floating-point arithmetic. Moreover, we show that preconditioning is important, not only for improving convergence but also reducing the risk of overflow.
△ Less
Submitted 3 March, 2021; v1 submitted 16 September, 2020;
originally announced September 2020.
-
White Paper from Workshop on Large-scale Parallel Numerical Computing Technology (LSPANC 2020): HPC and Computer Arithmetic toward Minimal-Precision Computing
Authors:
Roman Iakymchuk,
Daichi Mukunoki,
Artur Podobas,
Fabienne Jézéquel,
Toshiyuki Imamura,
Norihisa Fujita,
Jens Huthmann,
Shuhei Kudo,
Yiyu Tan,
Jens Domke,
Kai Torben Ohlhus,
Takeshi Fukaya,
Takeo Hoshi,
Yuki Murakami,
Maho Nakata,
Takeshi Ogita,
Kentaro Sano,
Taisuke Boku
Abstract:
In numerical computations, precision of floating-point computations is a key factor to determine the performance (speed and energy-efficiency) as well as the reliability (accuracy and reproducibility). However, precision generally plays a contrary role for both. Therefore, the ultimate concept for maximizing both at the same time is the minimal-precision computing through precision-tuning, which a…
▽ More
In numerical computations, precision of floating-point computations is a key factor to determine the performance (speed and energy-efficiency) as well as the reliability (accuracy and reproducibility). However, precision generally plays a contrary role for both. Therefore, the ultimate concept for maximizing both at the same time is the minimal-precision computing through precision-tuning, which adjusts the optimal precision for each operation and data. Several studies have been already conducted for it so far (e.g. Precimoniuos and Verrou), but the scope of those studies is limited to the precision-tuning alone. Hence, we aim to propose a broader concept of the minimal-precision computing system with precision-tuning, involving both hardware and software stack.
In 2019, we have started the Minimal-Precision Computing project to propose a more broad concept of the minimal-precision computing system with precision-tuning, involving both hardware and software stack. Specifically, our system combines (1) a precision-tuning method based on Discrete Stochastic Arithmetic (DSA), (2) arbitrary-precision arithmetic libraries, (3) fast and accurate numerical libraries, and (4) Field-Programmable Gate Array (FPGA) with High-Level Synthesis (HLS).
In this white paper, we aim to provide an overview of various technologies related to minimal- and mixed-precision, to outline the future direction of the project, as well as to discuss current challenges together with our project members and guest speakers at the LSPANC 2020 workshop; https://www.r-ccs.riken.jp/labs/lpnctrt/lspanc2020jan/.
△ Less
Submitted 11 April, 2020; v1 submitted 9 April, 2020;
originally announced April 2020.
-
Effect of Mixed Precision Computing on H-Matrix Vector Multiplication in BEM Analysis
Authors:
Rise Ooi,
Takeshi Iwashita,
Takeshi Fukaya,
Akihiro Ida,
Rio Yokota
Abstract:
Hierarchical Matrix (H-matrix) is an approximation technique which splits a target dense matrix into multiple submatrices, and where a selected portion of submatrices are low-rank approximated. The technique substantially reduces both time and space complexity of dense matrix vector multiplication, and hence has been applied to numerous practical problems.
In this paper, we aim to accelerate the…
▽ More
Hierarchical Matrix (H-matrix) is an approximation technique which splits a target dense matrix into multiple submatrices, and where a selected portion of submatrices are low-rank approximated. The technique substantially reduces both time and space complexity of dense matrix vector multiplication, and hence has been applied to numerous practical problems.
In this paper, we aim to accelerate the H-matrix vector multiplication by introducing mixed precision computing, where we employ both binary64 (FP64) and binary32 (FP32) arithmetic operations. We propose three methods to introduce mixed precision computing to H-matrix vector multiplication, and then evaluate them in a boundary element method (BEM) analysis. The numerical tests examine the effects of mixed precision computing, particularly on the required simulation time and rate of convergence of the iterative (BiCG-STAB) linear solver. We confirm the effectiveness of the proposed methods.
△ Less
Submitted 30 October, 2019;
originally announced November 2019.
-
Hierarchical Block Multi-Color Ordering: A New Parallel Ordering Method for Vectorization and Parallelization of the Sparse Triangular Solver in the ICCG Method
Authors:
Takeshi Iwashita,
Senxi Li,
Takeshi Fukaya
Abstract:
In this paper, we propose a new parallel ordering method to vectorize and parallelize the sparse triangular solver, which is called hierarchical block multi-color ordering. In this method, the parallel forward and backward substitutions can be vectorized while preserving the advantages of block multi-color ordering, that is, fast convergence and fewer thread synchronizations. To evaluate the propo…
▽ More
In this paper, we propose a new parallel ordering method to vectorize and parallelize the sparse triangular solver, which is called hierarchical block multi-color ordering. In this method, the parallel forward and backward substitutions can be vectorized while preserving the advantages of block multi-color ordering, that is, fast convergence and fewer thread synchronizations. To evaluate the proposed method in a parallel ICCG (Incomplete Cholesky Conjugate Gradient) solver, numerical tests were conducted using five test matrices on three types of computational nodes. The numerical results indicate that the proposed method outperforms the conventional block and nodal multi-color ordering methods in 13 out of 15 test cases, which confirms the effectiveness of the method.
△ Less
Submitted 2 August, 2019;
originally announced August 2019.
