一句话：提出任务式张量编程模型 Cypress 及其编译器，以顺序语义屏蔽异步数据搬运与同步，性能比肩专家手写 CUDA。
现代处理器中面向特定领域的固定功能单元日益普遍，NVIDIA Hopper GPU 每个计算单元内含异步数据搬运单元 TMA 与异步矩阵乘单元 Tensor Core，高效使用它们要求程序员编写 warp-specialized 内核并手工编排生产者—消费者流水线，编程复杂度高。为降低这一负担，作者提出 Cypress：一种具有顺序语义的任务式编程模型，程序由作用于张量的任务（task）构成，不包含任何通信与同步；程序通过 mapping specification 绑定到目标机，描述任务运行位置与张量应物化于何种内存。论文给出将该语言降级为 CUDA 程序的编译器架构。实验中 GEMM 达到 cuBLAS 的 0.88x–1.06x，Flash Attention 达到当前最佳实现的 0.80x–0.98x，且应用代码中完全消除了显式数据搬运与异步计算。
Domain-specific, fixed-function units are becoming increasingly common in modern processors. As the computational demands of applications evolve, the capabilities and programming interfaces of these fixed-function units continue to change. NVIDIA’s Hopper GPU architecture contains multiple fixed-function units per compute unit, including an asynchronous data movement unit (TMA) and an asynchronous matrix multiplication unit (Tensor Core). Efficiently utilizing these units requires a fundamentally different programming style than previous architectures; programmers must now develop warp-specialized kernels that orchestrate producer consumer pipelines between the asynchronous units. To manage the complexity of programming these new architectures, we introduce Cypress, a task-based programming model with sequential semantics. Cypress programs are a set of designated functions called tasks that operate on tensors and are free of communication and synchronization. Cypress programs are bound to the target machine through a mapping specification that describes where tasks should run and in which memories tensors should be materialized. We present a compiler architecture that lowers Cypress programs into CUDA programs that perform competitively with expert-written codes. Cypress achieves 0.88x-1.06x the performance of cuBLAS on GEMM, and between 0.80x-0.98x the performance of the currently best-known Flash Attention implementation while eliminating all aspects of explicit data movement and asynchronous computation from application code.
@article{paperbot3070,
title = {Task-Based Tensor Computations on Modern GPUs},
author = {Rohan Yadav and Michael Garland and Alex Aiken and Michael Bauer},
journal = {Proceedings of the ACM on Programming Languages},
volume = {9},
number = {PLDI},
year = {2025},
doi = {10.1145/3729262}
}