一句话：提出 FuseFlow，首个支持稀疏算子通用跨表达式融合的编译器，将 PyTorch 稀疏模型映射到可重构数据流架构并探索融合粒度。
针对稀疏深度学习模型与专用数据流硬件的效率需求，FuseFlow 将 PyTorch 稀疏模型编译为面向可重构数据流架构的融合稀疏数据流图。它是首个支持稀疏算子通用跨表达式融合的编译器，并集成并行化、数据流排序与稀疏分块等优化，后端采用周期精确数据流模拟器进行微架构分析。作者在四个真实稀疏机器学习应用上开展设计空间探索，表明端到端全融合并非总是最优，融合粒度取决于模型；其启发式可识别并剪除次优配置。对 GPT-3 的 BigBird 块稀疏注意力，取得约 2.7 倍于未融合基线的加速。
As deep learning models scale, sparse deep learning (DL) models that exploit sparsity in weights, activations, or inputs and specialized dataflow hardware have emerged as powerful solutions to address efficiency. We propose FuseFlow, a compiler that converts sparse machine learning models written in PyTorch to fused sparse dataflow graphs for reconfigurable dataflow architectures (RDAs). FuseFlow is the first compiler to support general cross-expression fusion of sparse operations. In addition to fusion across kernels (expressions), FuseFlow also supports optimizations like parallelization, dataflow ordering, and sparsity blocking. It targets a cycle-accurate dataflow simulator for microarchitectural analysis of fusion strategies. We use FuseFlow for design-space exploration across four real-world machine learning applications with sparsity, showing that full fusion (entire cross-expression fusion across all computation in an end-to-end model) is not always optimal for sparse models—fusion granularity depends on the model itself. FuseFlow also provides a heuristic to identify and prune suboptimal configurations. Using FuseFlow, we achieve performance improvements, including a ~2.7x speedup over an unfused baseline for GPT-3 with BigBird block-sparse attention.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2511-04768,
title = {FuseFlow: A Fusion-Centric Compilation Framework for Sparse Deep Learning on Streaming Dataflow},
author = {Rubens Lacouture and Nathan Zhang and Ritvik Sharma and Marco Siracusa and Fredrik Kjolstad and Kunle Olukotun and Olivia Hsu},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790165}
}