-
From array algebra to energy efficiency on GPUs: Data and hardware shapes with dimension-lifting to optimize memory-processor layouts
Authors:
Lenore M. R. Mullin
Abstract:
We present a new formulation for parallel matrix multiplication (MM) to out-perform the standard row-column code design. This algorithm is formulated in the MoA formalism (A Mathematics of Arrays) and combines an array view of hardware (dimension-lifting) to extend indexing to physical memory/processing units, with a contiguous data layout derived from static transformations. This view of a hardwa…
▽ More
We present a new formulation for parallel matrix multiplication (MM) to out-perform the standard row-column code design. This algorithm is formulated in the MoA formalism (A Mathematics of Arrays) and combines an array view of hardware (dimension-lifting) to extend indexing to physical memory/processing units, with a contiguous data layout derived from static transformations. This view of a hardware-software model is thus a bridging model in the sense of Valiant's BSP. OpenACCcode was derived from the MoA expressions's normal form, producing optimal block sizes using the static information of types and shapes. Experiments were run on Nvidia V100 GPUs and reveal energy consumption which is quadratic in N, i.e. linear in the size of matrix. More generally this approach may be an ideal way of formulating, optimizing, and map** array algorithms to embedded hardware. This work builds upon recently published results of NREL scientists.
.
△ Less
Submitted 19 June, 2023;
originally announced June 2023.
-
Tensors Come of Age: Why the AI Revolution will help HPC
Authors:
John L. Gustafson,
Lenore M. Mullin
Abstract:
This article discusses how the automation of tensor algorithms, based on A Mathematics of Arrays and Psi Calculus, and a new way to represent numbers, Unum Arithmetic, enables mechanically provable, scalable, portable, and more numerically accurate software.
This article discusses how the automation of tensor algorithms, based on A Mathematics of Arrays and Psi Calculus, and a new way to represent numbers, Unum Arithmetic, enables mechanically provable, scalable, portable, and more numerically accurate software.
△ Less
Submitted 26 September, 2017;
originally announced September 2017.
-
Tensors and n-d Arrays:A Mathematics of Arrays (MoA), psi-Calculus and the Composition of Tensor and Array Operations
Authors:
Lenore M. Mullin,
James E. Raynolds
Abstract:
The Kronecker product is a key algorithm and is ubiquitous across the physical, biological, and computation social sciences. Thus considerations of optimal implementation are important. The need to have high performance and computational reproducibility is paramount. Moreover, due to the need to compose multiple Kronecker products, issues related to data structures, layout and indexing algebra r…
▽ More
The Kronecker product is a key algorithm and is ubiquitous across the physical, biological, and computation social sciences. Thus considerations of optimal implementation are important. The need to have high performance and computational reproducibility is paramount. Moreover, due to the need to compose multiple Kronecker products, issues related to data structures, layout and indexing algebra require a new look at an old problem. This paper discusses the outer product/tensor product and a special case of the tensor product: the Kronecker product, along with optimal implementation when composed, and mapped to complex processor/memory hierarchies. We discuss how the use of ``A Mathematics of Arrays" (MoA), and the psi-Calculus, (a calculus of indexing with shapes), provides optimal, verifiable, reproducible, scalable, and portable implementations of both hardware and software.
△ Less
Submitted 4 July, 2009;
originally announced July 2009.
-
A generalized inner and outer product of arbitrary multi-dimensional arrays using A Mathematics of Arrays (MoA)
Authors:
James E. Raynolds,
Lenore M. Mullin
Abstract:
An algorithm has been devised to compute the inner and outer product between two arbitrary multi-dimensional arrays A and B in a single piece of code. It was derived using A Mathematics of Arrays (MoA) and the $ψ$-calculus. Extensive tests of the new algorithm are presented for running in sequential as well as OpenMP multiple processor modes.
An algorithm has been devised to compute the inner and outer product between two arbitrary multi-dimensional arrays A and B in a single piece of code. It was derived using A Mathematics of Arrays (MoA) and the $ψ$-calculus. Extensive tests of the new algorithm are presented for running in sequential as well as OpenMP multiple processor modes.
△ Less
Submitted 4 July, 2009;
originally announced July 2009.
