Control Flow Management in Modern GPUs
Authors:
Mojtaba Abaie Shoushtary,
Jordi Tubella Murgadas,
Antonio Gonzalez
Abstract:
In GPUs, the control flow management mechanism determines which threads in a warp are active at any point in time. This mechanism monitors the control flow of scalar threads within a warp to optimize thread scheduling and plays a critical role in the utilization of execution resources. The control flow management mechanism can be controlled or assisted by software through instructions. However, GP…
▽ More
In GPUs, the control flow management mechanism determines which threads in a warp are active at any point in time. This mechanism monitors the control flow of scalar threads within a warp to optimize thread scheduling and plays a critical role in the utilization of execution resources. The control flow management mechanism can be controlled or assisted by software through instructions. However, GPU vendors do not disclose details about their compiler, ISA, or hardware implementations. This lack of transparency makes it challenging for researchers to understand how the control flow management mechanism functions, is implemented, or is assisted by software, which is crucial when it significantly affects their research. It is also problematic for performance modeling of GPUs, as one can only rely on traces from real hardware for control flow and cannot model or modify the functionality of the mechanism altering it.
This paper addresses this issue by defining a plausible semantic for control flow instructions in the Turing native ISA based on insights gleaned from experimental data using various benchmarks. Based on these definitions, we propose a low-cost mechanism for efficient control flow management named Hanoi. Hanoi ensures correctness and generates a control flow that is very close to real hardware. Our evaluation shows that the discrepancy between the control flow trace of real hardware and our mechanism is only 1.03% on average. Furthermore, when comparing the Instructions Per Cycle (IPC) of GPUs employing Hanoi with the native control flow management of actual hardware, the average difference is just 0.19%.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
A Lightweight, Compiler-Assisted Register File Cache for GPGPU
Authors:
Mojtaba Abaie Shoushtary,
Jose Maria Arnau,
Jordi Tubella Murgadas,
Antonio Gonzalez
Abstract:
Modern GPUs require an enormous register file (RF) to store the context of thousands of active threads. It consumes considerable energy and contains multiple large banks to provide enough throughput. Thus, a RF caching mechanism can significantly improve the performance and energy consumption of the GPUs by avoiding reads from the large banks that consume significant energy and may cause port conf…
▽ More
Modern GPUs require an enormous register file (RF) to store the context of thousands of active threads. It consumes considerable energy and contains multiple large banks to provide enough throughput. Thus, a RF caching mechanism can significantly improve the performance and energy consumption of the GPUs by avoiding reads from the large banks that consume significant energy and may cause port conflicts.
This paper introduces an energy-efficient RF caching mechanism called Malekeh that repurposes an existing component in GPUs' RF to operate as a cache in addition to its original functionality. In this way, Malekeh minimizes the overhead of adding a RF cache to GPUs. Besides, Malekeh leverages an issue scheduling policy that utilizes the reuse distance of the values in the RF cache and is controlled by a dynamic algorithm. The goal is to adapt the issue policy to the runtime program characteristics to maximize the GPU's performance and the hit ratio of the RF cache. The reuse distance is approximated by the compiler using profiling and is used at run time by the proposed caching scheme. We show that Malekeh reduces the number of reads to the RF banks by 46.4% and the dynamic energy of the RF by 28.3%. Besides, it improves performance by 6.1% while adding only 2KB of extra storage per core to the baseline RF of 256KB, which represents a negligible overhead of 0.78%.
△ Less
Submitted 26 October, 2023;
originally announced October 2023.
