一句话：提出 AutoChecker，仅凭规则描述与测试套件，用 LLM 迭代生成代码检查器，平均测试通过率达 82.28%。
针对开发者定制静态代码检查器时面临规则逻辑复杂、检查器框架 API 庞大难以上手的问题，本文提出 AutoChecker：一种由 LLM 驱动的自动化检查器生成方法，输入仅为一条规则描述和一个测试套件。为覆盖完整的检查逻辑，AutoChecker 采取增量式更新策略，每次迭代只聚焦解决一个选定的测试用例，从而逐步补全检查器逻辑；为获取准确的 API 知识，每次迭代还引入细粒度的逻辑引导式 API 上下文检索——先将检查逻辑分解为一系列子操作，再为每个子操作检索与之相关的检查器框架 API 上下文。评估中，作者使用多种 LLM，让 AutoChecker、五个基线方法与三种消融方法为随机选取的 20 条 PMD 规则生成检查器。实验结果显示 AutoChecker 在所有有效性指标上均显著优于对比方法，平均测试通过率为 82.28%；并且在真实项目上应用时，其生成的检查器性能与官方检查器相当。
With the rising demand for code quality assurance, developers are not only utilizing existing static code checkers but also seeking custom checkers to satisfy their specific needs. Nowadays, various code-checking frameworks provide extensive checker customization interfaces to meet this need. However, both the complex checking logic of rules and massive API usages of large-scale checker frameworks make this task challenging. To this end, automated code checker generation is anticipated to ease the burden of checker development. In this paper, we propose AutoChecker, an innovative LLM-powered approach that can write code checkers automatically based on only a rule description and a test suite. To achieve comprehensive checking logic, AutoChecker incrementally updates the checker’s logic by focusing on solving one selected case each time. To obtain precise API knowledge, during each iteration, it leverages fine-grained logic-guided API-context retrieval, where it first decomposes the checking logic into a series of sub-operations and then retrieves checker-related API-contexts for each sub-operation. For evaluation, we apply AutoChecker, five baselines, and three ablation methods using multiple LLMs to generate checkers for 20 randomly selected PMD rules. Experimental results show that AutoChecker significantly outperforms others across all effectiveness metrics, with an average test pass rate of 82.28%. Additionally, checkers generated by AutoChecker show performance comparable to that of the official checkers when applied to real-world projects.
@inproceedings{paperbot11506,
title = {Write Your Own Code Checker: An Automated Test-Driven Checker Development Approach with LLMs},
author = {Jun Liu and Yuanyuan Xie and Jiwei Yan and Jinhao Huang and Jun Yan and Jian Zhang},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3764560}
}