-
Fast Matrix Multiplication via Compiler-only Layered Data Reorganization and Intrinsic Lowering
Authors:
Braedy Kuzma,
Ivan Korostelev,
João P. L. de Carvalho,
José E. Moreira,
Christopher Barton,
Guido Araujo,
José Nelson Amaral
Abstract:
The resurgence of machine learning has increased the demand for high-performance basic linear algebra subroutines (BLAS), which have long depended on libraries to achieve peak performance on commodity hardware. High-performance BLAS implementations rely on a layered approach that consists of tiling and packing layers, for data (re)organization, and micro kernels that perform the actual computation…
▽ More
The resurgence of machine learning has increased the demand for high-performance basic linear algebra subroutines (BLAS), which have long depended on libraries to achieve peak performance on commodity hardware. High-performance BLAS implementations rely on a layered approach that consists of tiling and packing layers, for data (re)organization, and micro kernels that perform the actual computations. The creation of high-performance micro kernels requires significant development effort to write tailored assembly code for each architecture. This hand optimization task is complicated by the recent introduction of matrix engines by IBM's POWER10 MMA, Intel AMX, and Arm ME to deliver high-performance matrix operations. This paper presents a compiler-only alternative to the use of high-performance libraries by incorporating, to the best of our knowledge and for the first time, the automatic generation of the layered approach into LLVM, a production compiler. Modular design of the algorithm, such as the use of LLVM's matrix-multiply intrinsic for a clear interface between the tiling and packing layers and the micro kernel, makes it easy to retarget the code generation to multiple accelerators. The use of intrinsics enables a comprehensive performance study. In processors without hardware matrix engines, the tiling and packing delivers performance up to 22x (Intel), for small matrices, and more than 6x (POWER9), for large matrices, faster than PLuTo, a widely used polyhedral optimizer. The performance also approaches high-performance libraries and is only 34% slower than OpenBLAS and on-par with Eigen for large matrices. With MMA in POWER10 this solution is, for large matrices, over 2.6x faster than the vector-extension solution, matches Eigen performance, and achieves up to 96% of BLAS peak performance.
△ Less
Submitted 15 May, 2023;
originally announced May 2023.
-
A matrix math facility for Power ISA(TM) processors
Authors:
José E. Moreira,
Kit Barton,
Steven Battle,
Peter Bergner,
Ramon Bertran,
Puneeth Bhat,
Pedro Caldeira,
David Edelsohn,
Gordon Fossum,
Brad Frey,
Nemanja Ivanovic,
Chip Kerchner,
Vincent Lim,
Shakti Kapoor,
Tulio Machado Filho,
Silvia Melitta Mueller,
Brett Olsson,
Satish Sadasivam,
Baptiste Saleil,
Bill Schmidt,
Rajalakshmi Srinivasaraghavan,
Shricharan Srivatsan,
Brian Thompto,
Andreas Wagner,
Nelson Wu
Abstract:
Power ISA(TM) Version 3.1 has introduced a new family of matrix math instructions, collectively known as the Matrix-Multiply Assist (MMA) facility. The instructions in this facility implement numerical linear algebra operations on small matrices and are meant to accelerate computation-intensive kernels, such as matrix multiplication, convolution and discrete Fourier transform. These instructions h…
▽ More
Power ISA(TM) Version 3.1 has introduced a new family of matrix math instructions, collectively known as the Matrix-Multiply Assist (MMA) facility. The instructions in this facility implement numerical linear algebra operations on small matrices and are meant to accelerate computation-intensive kernels, such as matrix multiplication, convolution and discrete Fourier transform. These instructions have led to a power- and area-efficient implementation of a high throughput math engine in the future POWER10 processor. Performance per core is 4 times better, at constant frequency, than the previous generation POWER9 processor. We also advocate the use of compiler built-ins as the preferred way of leveraging these instructions, which we illustrate through case studies covering matrix multiplication and convolution.
△ Less
Submitted 7 April, 2021;
originally announced April 2021.
-
Brazilian elections: voting for a scaling democracy
Authors:
R. N. Costa Filho,
M. P. Almeida,
J. E. Moreira,
J. S. Andrade Jr
Abstract:
The proportional elections held in Brazil in 1998 and 2002 display identical statistical signatures. In particular, the distribution of votes among candidates includes a power-law regimen. We suggest that the rationale behind this robust scaling invariance is a multiplicative process in which the voter's choice for a candidate is governed by a product of probabilities.
The proportional elections held in Brazil in 1998 and 2002 display identical statistical signatures. In particular, the distribution of votes among candidates includes a power-law regimen. We suggest that the rationale behind this robust scaling invariance is a multiplicative process in which the voter's choice for a candidate is governed by a product of probabilities.
△ Less
Submitted 12 November, 2002; v1 submitted 11 November, 2002;
originally announced November 2002.
