Warped-preexecution: A GPU pre-execution approach for improving latency hiding

Sangpil Lee, Won Woo Ro, Keunsoo Kim, Gunjae Koo, Myung Kuk Yoon, Murali Annavaram

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

32 Scopus citations


This paper presents a pre-execution approach for improving GPU performance, called P-mode (pre-execution mode). GPUs utilize a number of concurrent threads for hiding processing delay of operations. However, certain long-latency operations such as off-chip memory accesses often take hundreds of cycles and hence leads to stalls even in the presence of thread concurrency and fast thread switching capability. It is unclear if adding more threads can improve latency tolerance due to increased memory contention. Further, adding more threads increases on-chip storage demands. Instead we propose that when a warp is stalled on a long-latency operation it enters P-mode. In P-mode, a warp continues to fetch and decode successive instructions to identify any independent instruction that is not on the long latency dependence chain. These independent instructions are then pre-executed. To tackle write-after-write and write-after-read hazards, during P-mode output values are written to renamed physical registers. We exploit the register file underutilization to re-purpose a few unused registers to store the P-mode results. When a warp is switched from P-mode to normal execution mode it reuses pre-executed results by reading the renamed registers. Any global load operation in P-mode is transformed into a pre-load which fetches data into the L1 cache to reduce future memory access penalties. Our evaluation results show 23% performance improvement for memory intensive applications, without negatively impacting other application categories.

Original languageEnglish
Title of host publicationProceedings of the 2016 IEEE International Symposium on High-Performance Computer Architecture, HPCA 2016
PublisherIEEE Computer Society
Number of pages13
ISBN (Electronic)9781467392112
StatePublished - 1 Apr 2016
Event22nd IEEE International Symposium on High Performance Computer Architecture, HPCA 2016 - Barcelona, Spain
Duration: 12 Mar 201616 Mar 2016

Publication series

NameProceedings - International Symposium on High-Performance Computer Architecture
ISSN (Print)1530-0897


Conference22nd IEEE International Symposium on High Performance Computer Architecture, HPCA 2016

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. NRF-2015R1A2A2A01008281), and by the following grants: DARPA-PERFECT-HR0011-12-2-0020 and NSF-CAREER-0954211, NSF-0834798.

Publisher Copyright:
© 2016 IEEE.


Dive into the research topics of 'Warped-preexecution: A GPU pre-execution approach for improving latency hiding'. Together they form a unique fingerprint.

Cite this