一句话：纯软件运行时系统 RPG²，在线剖析程序并注入、调优预取指令，最高加速 2.15×，且能在预取有害时回滚。
数据缓存预取是克服缓存层次限制的经典优化，但预取何数据、何时预取难以确定：过早会驱逐有用数据，过晚则无法避免缺失，还要与程序的常规需求访存竞争有限的片外带宽；实验表明其效果还依赖程序结构、输入与处理器微架构及其交互。为此作者设计了 RPG²，一个面向运行中 C/C++ 程序的纯软件在线预取注入与调优系统：它对运行中的程序进行剖析，注入预取指令，并依据性能反馈持续调优预取。在数十个输入上的评估显示，RPG² 可获得最高 2.15× 的加速，与最佳的基于剖析的预取编译器相当；更重要的是，当预取反而有害时它能回滚到原始代码，这是静态编译器无法做到的，从而在保留收益的同时提升了预取的鲁棒性。
Data cache prefetching is a well-established optimization to overcome the limits of the cache hierarchy and keep the processor pipeline fed with data. In principle, accurate, well-timed prefetches can sidestep the majority of cache misses and dramatically improve performance. In practice, however, it is challenging to identify which data to prefetch and when to do so. In particular, data can be easily requested too early, causing eviction of useful data from the cache, or requested too late, failing to avoid cache misses. Competition for limited off-chip memory bandwidth must also be balanced between prefetches and a program's regular "demand" accesses. Due to these challenges, prefetching can both help and hurt performance, and the outcome can depend on program structure, decisions about what to prefetch and when to do it, and, as we demonstrate in a series of experiments, program input, processor microarchitecture, and their interaction as well. To try to meet these challenges, we have designed the RPG2 system for online prefetch injection and tuning. RPG2 is a pure-software system that operates on running C/C++ programs, profiling them, injecting prefetch instructions, and then tuning those prefetches to maximize performance. Across dozens of inputs, we find that RPG2 can provide speedups of up to 2.15×, comparable to the best profile-guided prefetching compilers, but can also respond when prefetching ends up being harmful and roll back to the original code - something that static compilers cannot. RPG2 improves prefetching robustness by preserving its performance benefits, while avoiding slowdowns.
@inproceedings{ZhangSPJKKPLD24,
title = {RPG
<sup>2</sup>
: Robust Profile-Guided Runtime Prefetch Generation},
author = {Yuxuan Zhang and Nathan Sobotka and Soyoon Park and Saba Jamilan and Tanvir Ahmed Khan and Baris Kasikci and Gilles A Pokam and Heiner Litz and Joseph Devietti},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620665.3640396}
}