一句话：提出 COMFUZZ，用深度学习生成并变异测试程序，聚焦编译器易错特性以提升模糊测试效率。
随机生成的测试用例难以高效触发现代编译器的复杂缺陷，使编译器模糊测试耗时且命中率低。COMFUZZ 的核心观察是不同编译器实现中开发者常犯重复性错误，因此可利用某语言已暴露缺陷的测试用例来测试新编译器。它首先用深度学习从 GitHub 开源项目学习测试程序生成器，再通过精心设计的变异规则提升测试用例的覆盖率和缺陷暴露能力。作者在 11 个 JavaScript 和 Java 编译器上评估，260 小时自动化测试发现 9 个编译器中 33 个独特缺陷，其中 29 个被确认、22 个已修复；与 8 个已有模糊测试器相比，24 小时内发现的缺陷至少多 1.5 倍。
Random test case generation, or fuzzing, is a viable means for uncovering compiler bugs. Unfortunately, compiler fuzzing can be time-consuming and inefficient with purely randomly generated test cases due to the complexity of modern compilers. We present COMFUZZ, a focused compiler fuzzing framework. COMFUZZ aims to improve compiler fuzzing efficiency by focusing on testing components and language features that are likely to trigger compiler bugs. Our key insight is human developers tend to make common and repeat errors across compiler implementations; hence, we can leverage the previously reported buggy-exposing test cases of a programming language to test a new compiler implementation. To this end, COMFUZZ employs deep learning to learn a test program generator from open-source projects hosted on GitHub. With the machine-generated test programs in place, COMFUZZ then leverages a set of carefully designed mutation rules to improve the coverage and bug-exposing capabilities of the test cases. We evaluate COMFUZZ on 11 compilers for JS and Java programming languages. Within 260 hours of automated testing runs, we discovered 33 unique bugs across nine compilers, of which 29 have been confirmed and 22, including an API documentation defect, have already been fixed by the developers. We also compared COMFUZZ to eight prior fuzzers on four evaluation metrics. In a 24-hour comparative test, COMFUZZ uncovers at least 1.5× more bugs than the state-of-the-art baselines.
@inproceedings{YeHTFTZQ023,
title = {A Generative and Mutational Approach for Synthesizing Bug-Exposing Test Cases to Guide Compiler Fuzzing},
author = {Guixin Ye and Tianmin Hu and Zhanyong Tang and Zhenye Fan and Shin Hwei Tan and Bo Zhang and Wenxiang Qian and Zheng Wang},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3616332}
}