一句话：提出 AGIO，为 GPU 线程提供直接异步 I/O 的 API 与运行时，将 I/O 发起与完成解耦以隐藏 SSD 访问延迟。
GPU 主要依赖同步的编程与执行模型，而 GPU 线程直接访问 SSD 时，阻塞式/同步 I/O 会带来显著延迟；现有软硬件也缺乏支持 GPU 线程直接发起非阻塞/异步操作以容忍微秒级 SSD 延迟的机制。本文提出 AGIO，提供 API 与运行时环境，让 GPU 线程在不涉及 CPU 的情况下完全由 GPU 编排地执行异步 I/O。AGIO 在时间和空间上将 I/O 发起与完成解耦，使两者之间可插入有效计算以隐藏 I/O 延迟；对编译期已知的访问模式，可像预取一样提前发起 I/O，性能比同步方案提升 65%；对数据依赖访问，即使可重叠计算很少，也能让线程越过 I/O 发起去主动发起更多 I/O，提高 I/O 并行性并更充分利用带宽，性能提升 32%。AGIO 还使性能更自适应于数据集特征，无需程序员先验了解输入，并可在计算引擎数不到同步方案一半的 GPU 上达到或超过其性能。
GPUs rely on a largely synchronous programming and execution model. With increasing need to access data residing on SSDs, GPU threads can incur significant latencies for such accesses when using blocking/synchronous I/O mechanisms. There is little hardware/systems support today to perform non-blocking/asynchronous operations from GPU threads directly to tolerate microsecond level latencies incurred in SSD accesses. To fix this dichotomy, this paper presents the design, implementation and evaluation of AGIO, which provides APIs and a runtime environment for GPU threads to directly perform asynchronous I/O operations (fully GPU-orchestrated without CPU involvement). AGIO decouples, both in time and space, the I/O initiation from its completion, to allow useful computation in-between, in order to hide much of the I/O latency. This is particularly useful in applications with access patterns known at compile time, where similar to prefetching, AGIO I/Os can be introduced ahead of need, to yield 65% better performance than its synchronous counterpart. A non-intuitive benefit of AGIO, particularly in applications with data dependent accesses, is the ability to allow threads to proceed beyond I/O initiation, towards initiating more I/O, even if there is little compute to overlap. Such pro-active I/O issuance increases the I/O parallelism to more fully utilize underlying bandwidths, yielding 32% better performance than the synchronous alternative in data dependent executions. Decoupling initiation from completion makes AGIO more adaptive to dataset characteristics, with the programmer not needing a priori knowledge of inputs for effective performance. We also show that AGIO can meet (or better) the performance using a GPU with fewer than half the compute engines of its synchronous counterparts.
@inproceedings{HanSCMQH26,
title = {Asynchrony and GPUs: Bridging this Dichotomy for I/O with AGIO},
author = {Jihoon Han and Anand Sivasubramaniam and Chia-Hao Chang and Vikram Sharma Mailthody and Zaid Qureshi and Wen-Mei Hwu},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790130}
}