一句话：提出 Render，用动态区间算术与舍入误差分析估计最大舍入误差，区分可接受误差与实现缺陷。
深度学习程序常通过 kernel 级优化、并行训练和低精度算术获得性能提升，由此产生多个数学等价但数值行为不同的实现。浮点舍入误差会使这些实现的输出出现差异，当定制实现与参考实现的输出超出容差阈值时，开发者难以判断这是可接受的舍入误差还是实现 bug。本文提出 Render，通过估计最大舍入误差来对两个实现之间的数值误差进行分类。Render 将动态区间算术与舍入误差分析相结合，计算出可扩展且紧致的输出界。作者在真实问题上将 Render 与当前最先进的工具 SATIRE 以及高精度重执行基线进行对比，实验表明 Render 多识别出至少 25% 的错误，并相对 SATIRE 取得平均 19 倍的加速，从而帮助开发者更高效地调试与优化实现。
Deep learning programs are continually enhanced for improved performance through the use of kernel-level optimizations, parallel training, and low-precision arithmetic. These optimizations provide different implementations that are mathematically equivalent. Round-off error in floating-point computations can lead to differences in the outputs of these implementations, even when the mathematical equivalence holds. When the outputs of customized and reference implementations exceed the tolerance thresholds, it is difficult for developers to distinguish between acceptable round-off errors and implementation bugs. This paper proposes an approach called Render to classify the numerical errors between two implementations based on estimating the maximum round-off error. Render combines dynamic interval arithmetic and round-off error analysis to compute scalable and tight output bounds. We demonstrate the effectiveness of our method on real-world issues by comparing it with the state-of-the-art tool, SATIRE and a High-Precision Re-execution baseline. Experimental results show that our approach identifies at least 25% more errors and achieves an average speedup of 19× compared to SATIRE, enabling developers to debug and optimize implementations more efficiently.
DOI 原文 ·
@inproceedings{paperbot9455,
title = {When AllClose Fails: Round-Off Error Estimation for Deep Learning Programs},
author = {Qi Zhan and Xing Hu and Yuanyi Lin and Tongtong Xu and Xin Xia and Shanping Li},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00016}
}