一句话：提出 LLM 驱动的三阶段配置感知优化流水线，自动生成配置专用测试套件以检测配置错误。
软件配置在现代系统中至关重要，却频繁导致严重故障。现有静态分析与基于 LLM 的验证难以捕捉隐式语义约束并验证配置与代码的交互，而复用既有单元测试又缺乏配置针对性。为此，作者提出 CTForge——一个由 LLM 驱动的框架，自动生成面向配置的测试套件。其核心是三阶段配置感知优化流水线，利用参数语义改进 LLM 输出：先增强配置相关代码覆盖率以保证相关性，再在有效参数取值上验证断言以消除虚假断言，最后进行配置敏感性验证以筛选真正在语义上触发目标配置行为的测试。在五个大型系统上，CTForge 检出 79 个真实配置问题中的 63 个，误配置检测 F1 达 0.88，并发现六个此前未知的缺陷，表明结构化配置感知精化对 LLM 生成有效测试套件不可或缺。
Software configuration plays a critical role in modern software systems, yet it also frequently leads to severe failures. Existing approaches, whether static analysis or LLM-based validation, struggle to capture implicit semantic constraints and validate configuration-code interactions. Leveraging existing unit tests is also insufficient due to their lack of configuration-specific focus. Thus, we present CTForge, an LLM-powered framework that automatically generates configuration-specific test suites. Its core innovation is a three-stage configuration-aware optimization pipeline that moves beyond vanilla test generation, specifically using parameter semantics to improve LLM outputs. This optimization pipeline iteratively aligns tests through: (1)enhance configuration-related code coverage to ensure relevance, (2) assertion validation across valid parameter values to eliminate false assertions, and (3) configuration-sensitivity verification to select tests that semantically exercise the intended configuration behavior. Evaluated on five large-scale systems, CTForge detects 63 out of 79 real-world configuration issues and achieves an F1-score of 0.88 in misconfiguration detection. CTForge also uncovers six previously unknown defects. Our work demonstrates that structured configuration-aware refinement is essential for LLM to produce effective test suites.
@inproceedings{paperbot12515,
title = {CTForge: Automatically Generating Test Suites for Software Configuration},
author = {Yuanliang Zhang and Zhizheng Zheng and Shanshan Li and Zhouyang Jia and Chaopeng Luo and Liqian Chen and Zhenbang Chen and Ji Wang and Xiangke Liao},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3837484}
}