一句话：首个用组合测试探索优化序列空间的编译器测试方法，能检出更多由特定优化序列触发的缺陷。
编译器是现代软件的基础，其可靠性至关重要。GCC、LLVM 等现代编译器提供大量优化，经验研究表明多数缺陷出现在优化阶段，尤其与特定优化序列密切相关。然而，优化序列空间极其庞大，现有编译器测试方法难以有效探索这一方向。本文提出 CombCT，首次将组合测试引入编译器测试，以高效探索优化序列；同时引入动态优化序列适配、部分序列选择和多样性增强三项新技术以提升效果。作者在多个 GCC 与 LLVM 版本上开展实验，结果表明 CombCT 比当前最先进方法检测到更多由特定优化序列导致的唯一缺陷，并在最新版本中发现 14 个此前未知的缺陷，其中 13 个已被确认或修复。
Compilers, as fundamental software, are essential for various applications, making their reliability crucial. Modern compilers like GCC and LLVM offer numerous optimizations to improve performance. Empirical studies present that most bugs occur during compiler optimization, especially with specific optimization sequences, yet little attention has been paid to this in compiler testing. The vast optimization sequence space challenges existing methods, leaving this important research direction in its early stages and in need of effective solutions. In this work, we propose a novel and effective compiler testing approach dubbed CombCT, aimed at boosting compiler testing. CombCT is the first approach that leverages the power of combinatorial testing for compiler testing, making it be capable of effectively exploring optimization sequences. Besides, CombCT incorporates three new techniques, i.e., dynamic optimization sequence adaption, partial sequence selection, and diversity augmentation, to enhance its effectiveness. Extensive experiments on various versions of GCC and LLVM demonstrate that CombCT detects much more unique bugs, which are caused by certain optimization sequences, than current state-of-the-art methods, indicating its superiority. Moreover, CombCT has disclosed 14 previously unknown bugs in the latest versions of GCC and LLVM, 13 of which have been confirmed or fixed, showing its practical value.
@inproceedings{paperbot11945,
title = {CombCT: Compiler Testing via Combinatorial Testing},
author = {Chuan Luo and Shaoke Cui and Jiahao Yan and Junjie Chen and Chenyao Suo and Wei Wu and Chanjuan Liu and Chunming Hu},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3764548}
}