一句话:提出 ScenSCT,用 LLM 从检查器源码抽取检查场景并生成测试,暴露静态分析工具大量误报与漏报。
静态分析工具中每条检测规则对应一个专用代码检查器,检查器质量直接决定缺陷报告的有效性,因此需要全面测试。现有方法多依据高层规则描述生成测试用例,并把检查器当作黑盒,然而规则描述往往遗漏实现层的判定条件与检查场景。本文的关键洞见是:规则描述与检查器实现提供互补信息——实现给出具体检查逻辑,规则锚定预期语义。据此作者提出 ScenSCT:先将检查器源码翻译为统一的逻辑表示 CheckerIR,再通过逻辑分解与场景验证两类策略抽取具体检查场景;随后借助执行反馈引导的测试对齐实例化场景约束,并用场景不变式增强生成保持目标场景的变体,从而把场景转化为测试用例。在来自五个主流静态分析工具的 Java 检查器上评估,单工具设定下平均每个检查器抽取约 9 个场景、生成约 39 个测试,发现 270 个潜在问题(58 个误报、212 个漏报),总体精确率 86.4%;跨工具评估还揭示了共享检查器之间的 177 处能力差异。
In static analysis tools, each detection rule corresponds to a dedicated code checker. The checker's quality directly determines the validity of reported defects. Thus, comprehensive testing of these checkers is crucial. Existing checker testing approaches usually generate test cases from high-level rule descriptions while treating checkers as black boxes. However, these descriptions usually omit implementation-level conditions and checking scenarios. To bridge this gap, our key insight is that the rule description and checker implementation provide complementary information for comprehensive testing. The implementation supplies concrete checking logic, while the rule anchors the intended semantics. Building on this, we propose ScenSCT, a novel scenario-guided white-box checker testing approach powered by LLMs. ScenSCT first translates the checker's source code into a unified logic-based representation (CheckerIR) and applies both logic-decomposition and scenario-validation strategies to extract concrete checking scenarios. It then converts these scenarios into test cases through execution-feedback-guided test alignment to instantiate scenario constraints and scenario-invariant test augmentation to generate variants that preserve the target scenario. We evaluate ScenSCT on real-world Java checkers from five mainstream static analysis tools. In single-tool evaluation, ScenSCT extracts an average of ~9 scenarios and generates ~39 tests per checker, uncovering 270 potential issues (58 false positives and 212 false negatives) with an overall precision of 86.4%. Furthermore, cross-tool evaluation reveals 177 capability differences among shared checkers. Overall, ScenSCT offers a systematic testing method to improve the reliability of static analysis tools.
@inproceedings{paperbot12526,
title = {Every Scenario Matters: Scenario-Guided White-Box Testing for Static Code Checkers Powered by LLMs},
author = {Jun Liu and Liqiang Ji and Jiwei Yan and Linjie Pan and Jinsong Ren and Jun Yan and Jian Zhang},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834381}
}