paperbot · PL 论文追踪

全部论文 待读 收藏

P2 E-Test: E'er-Improving Test Suites

E-Test:不断改进的测试套件
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 4/10 · 该论文属于 LLM 辅助软件测试(动态分析)方向,与用户关注的 AI→PL(用 LLM 改进静态分析/形式化验证)在方法论上相邻,但既非静态分析/形式化验证,也不涉及浮点程序分析或贝叶斯程序分析,未命中任一核心交叉点。引用 2 inbox
Ketai Qiu, Luca Di Grazia, Leonardo Mariani, Mauro Pezzè
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:利用 LLM 从大规模生产执行场景中识别未被测试覆盖的场景,并生成新测试用例以增强测试套件

测试套件天然不完善,可不断加入新用例以提升质量与可靠性。但在大型、长期维护的套件中,找到覆盖现有套件之外执行场景的用例极为困难且耗费人力。本文提出 E-Test,旨在缩小测试套件探索的执行空间与生产实际执行之间的差距。E-Test 利用大语言模型(LLM)从大规模执行场景(如生产监控数据)中识别尚未被测试的场景,并生成执行这些场景的新测试用例来增强套件。在 1,975 个执行场景的数据集上,E-Test 识别未测试场景的 F1 达 0.55,显著优于回归测试与现场测试方法(最高 0.34)及原生 LLM(最高 0.39),减少了套件维护的人工投入。

问题
测试套件无法覆盖生产环境中出现的所有执行场景,而人工从大规模执行数据中发现尚未被测试的场景并编写相应测试用例极为困难且成本高昂。
方法
E-Test 利用大语言模型从大规模执行场景集合(如生产监控数据)中识别当前测试套件未充分覆盖的场景,并自动生成执行这些场景的新测试用例以扩充测试套件。
评估
在包含 1,975 个执行场景的数据集上评估,E-Test 识别未测试场景的 F1 达到 0.55,而现有回归测试与现场测试方法最高为 0.34,原生 LLM 最高为 0.39。
原文摘要(Abstract)

Test suites are inherently imperfect, and testers can always enrich a suite with new test cases that improve its quality and, consequently, the reliability of the target software system. However, finding test cases that explore execution scenarios beyond the scope of an existing suite can be extremely challenging and labor-intensive, particularly when managing large test suites over extended periods. In this paper, we propose E-Test, an approach that reduces the gap between the execution space explored with a test suite and the executions experienced in production. E-Test (i) identifies executions that have not yet been tested from large sets of scenarios, such as those monitored during intensive production usage, and (ii) generates new test cases that enhance the test suite. E-Test leverages Large Language Models (LLMs) to pinpoint scenarios that the current test suite does not adequately cover, and augments the suite with test cases that execute these scenarios. Our evaluation on a dataset of 1,975 execution scenarios demonstrates that E-Test identifies not-yet-tested execution scenarios significantly better than state-of-the-art approaches. While existing regression testing and field testing approaches for this task achieve a maximum F1-score of 0.34, and vanilla LLMs achieve a maximum F1-score of 0.39, E-Test reaches 0.55. These results highlight the impact of E-Test in enhancing test suites by effectively targeting not-yet-tested execution scenarios and reducing manual effort required for maintaining test suites.

链接与引用

DOI 原文 · arXiv · DBLP

BibTeX
@inproceedings{abs-2510-19860,
  title = {E-Test: E'er-Improving Test Suites},
  author = {Ketai Qiu and Luca Di Grazia and Leonardo Mariani and Mauro Pezzè},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3773126}
}