一句话：提出基于括号掩码与 LLM 填充的 Rust 编译器模糊测试方法，发现 27 个确认缺陷。
Rust 编译器可靠性对关键系统至关重要，但为其生成有效测试程序十分困难：Rust 语法复杂、约束严格，而直接使用大语言模型生成程序常产生大量无效用例。本文受历史 issue 中触发过编译器 bug 的测试代码启发，提出 clozeMask 策略：从历史 issue 报告中提取测试代码，识别并掩码具有特定结构的代码片段，再利用 LLM 填充被掩码部分，从而合成既保留触发 bug 能力、又具备新结构的新测试程序。作者实现原型 ClozeMaster，对 rustc 与 mrustc 进行模糊测试，确认 27 个 bug，其中 10 个已被开发者修复；实验还表明其在代码覆盖率和测试有效性上优于现有 fuzzer。
Ensuring the reliability of the Rust compiler is of paramount importance, given increasing adoption of Rust for critical systems development, due to its emphasis on memory and thread safety. However, generating valid test programs for the Rust compiler poses significant challenges, given Rust's complex syntax and strict requirements. With the growing popularity of large language models (LLMs), much research in software testing has explored using LLMs to generate test cases. Still, directly using LLMs to generate Rust programs often results in a large number of invalid test cases. Existing studies have indicated that test cases triggering historical compiler bugs can assist in software testing. Our investigation into Rust compiler bug issues supports this observation. Inspired by existing work and our empirical research, we introduce a bracket-based masking and filling strategy called clozeMask. The clozeMask strategy involves extracting test code from historical issue reports, identifying and masking code snippets with specific structures, and using an LLM to fill in the masked portions for synthesizing new test programs. This approach harnesses the generative capabilities of LLMs while retaining the ability to trigger Rust compiler bugs. It enables comprehensive testing of the compiler's behavior, particularly exploring edge cases. We implemented our approach as a prototype ClozeMaster. ClozeMaster has identified 27 confirmed bugs for rustc and mrustc, of which 10 have been fixed by developers. Furthermore, our experimental results indicate that ClozeMaster outperforms existing fuzzers in terms of code coverage and effectiveness.
@inproceedings{paperbot8921,
title = {Clozemaster: Fuzzing Rust Compiler by Harnessing Llms for Infilling Masked Real Programs},
author = {Hongyan Gao and Yibiao Yang and Maolin Sun and Jiangchang Wu and Yuming Zhou and Baowen Xu},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00175}
}