一句话:提出 HirGen,通过生成多样化计算图与高层 IR 并设计覆盖率准则和测试预言,暴露 DL 编译器高层优化阶段缺陷。
深度学习编译器被广泛用于优化模型以适配多样硬件,其质量直接影响编译后模型的质量。近期缺陷研究表明,高层中间表示(IR)优化是最易出错的编译阶段,相关 bug 占 44.92%。然而,现有测试技术未考虑高层优化特性,难以暴露该阶段缺陷。为此,本文提出 HirGen,一种面向高层 IR 优化的自动化测试技术:它包含三种覆盖率准则以生成多样且有效的计算图,利用高层 IR 语言特性生成多样 IR,并采用三个测试预言,其中两个受蜕变测试与差分测试启发。在 TVM 上,HirGen 发现 21 个 bug,17 个被确认,12 个被修复;与四种可覆盖高层优化阶段的先进 DL 编译器模糊测试基线相比,HirGen 在 48 小时内检出 10 个基线无法发现的崩溃与不一致,并评估了所提覆盖率准则和测试预言的有效性。
Deep Learning (DL) compilers are widely adopted to optimize advanced DL models for efficient deployment on diverse hardware. Their quality has a profound effect on the quality of compiled DL models. A recent bug study shows that the optimization of high-level intermediate representations (IRs) is the most error-prone compilation stage and bugs in this stage account for 44.92% of the whole collected ones. However, existing testing techniques do not consider the features related to high-level optimization (e.g., the high-level IR), and are therefore weak in exposing bugs at this stage. To bridge this gap, we propose HirGen, an automated testing technique that effectively exposes coding mistakes in the optimization of high-level IRs. The design of HirGen includes 1) three coverage criteria to generate diverse and valid computational graphs; 2) the use of the high-level IR’s language features to generate diverse IRs; 3) three test oracles of which two are inspired by metamorphic testing and differential testing. HirGen has successfully detected 21 bugs that occur at TVM, with 17 bugs confirmed and 12 fixed. Further, we construct four baselines using state-of-the-art DL compiler fuzzers that can cover the high-level optimization stage. Our experiment results show that HirGen can detect 10 crashes and inconsistencies that cannot be detected by the baselines in 48 hours. We also evaluate the usefulness of our proposed coverage criteria and test oracles.