一句话：提出级联规约的形式化分析与增量计算形式，并实现自动融合与内核生成的 RedFuser 框架，在 AI 加速器上最高加速 2–5 倍。
算子融合是 AI 模型部署中的关键优化，但现有 AI 编译器难以自动融合具有循环间数据依赖的级联规约操作，如注意力机制中的 safe softmax 后接 GEMM。手工融合策略缺乏通用性。本文提出分析级联规约的形式化理论方法，将其融合为单循环并引入增量计算形式；基于此设计 RedFuser 框架，可自动识别受支持的级联规约模式并生成优化融合内核。实验表明 RedFuser 能成功融合多种工作负载，相较当前最优 AI 编译器取得最高 2–5 倍加速，并达到高度优化的手写内核性能。
Operator fusion, as a key performance optimization technique in the deployment of AI models, significantly improves execution efficiency and has been widely adopted in modern AI compilers. However, for cascaded reduction operations involving multiple loops with inter-loop data dependencies, such as the safe softmax followed by GEMM within attention mechanisms, existing compilers lack effective automated fusion and kernel generation capabilities. Although some works have addressed specific instances through hand-crafted fusion strategies, their solutions are limited in generality and difficult to extend to other similar structures. Given the prevalence of such computational patterns in deep learning models, there remains significant untapped potential in achieving general and automated fusion optimization. In this paper, we present a formal theoretical methodology for analyzing cascaded reductions which can fuse them into a single loop and introduce an incremental computation form. Based on this methodology, we design Red uction Fuser (RedFuser), a framework that automatically identifies supported cascaded reduction patterns and generates optimized fused kernels. Experiments show that RedFuser successfully fuses diverse workloads, achieving up to 2× to 5× speedup over state-of-the-art AI compilers and matching the performance of highly optimized hand-written kernels.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{TangLWSLXZL26,
title = {RedFuser: An Automatic Operator Fusion Framework for Cascaded Reductions on AI Accelerators},
author = {Xinsheng Tang and Yangcheng Li and Nan Wang and Zhiyi Shu and Xingyu Ling and Junna Xing and Peng Zhou and Qiang Liu},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790209}
}