一句话：在汽车工业场景中评估生成式 AI 能否从非形式化测试用例规范生成可执行测试脚本。
软件测试需要自动化支持，但生成式 AI 能否生成测试用例仍不明确。本文与一家大型汽车制造商合作，研究生成式 AI 能否在工业软件应用中产生测试用例。具体而言，输入是 IBM 工业测试管理工具 Rational Quality Manager 中定义的非形式化测试用例规范，输出是 Tracetronic 的 ecu.test 工具所支持的可执行测试脚本（ecu.test 包）。作者使用生成式 AI 完成从非形式化描述到测试脚本的转换，并进一步分析了提示设计、生成式 AI 模型选择以及上下文准确性对该方案有效性的影响。结果表明，在相当数量的案例中，生成式 AI 能够生成正确或接近正确的测试脚本，作者也据此对结果进行了反思。
Engineers need automated support for software testing. Generative AI is a novel technology for generating new content; however, its applicability for test case generation is still unclear. This work considers the following question: Can generative AI produce test cases in industrial software applications? We framed our question in the automotive domain. We performed our evaluation in collaboration with a large automotive manufacturer to assess to what extent generative AI can produce test cases (a.k.a. test scripts) from informal test case specifications. We considered 1) informal test case specifications defined in Rational Quality Manager, an industrial test management tool from IBM, and 2) executable test scripts specified as ecu.test packages supported by the ecu.test tool from Tracetronic. We used generative AI to produce the test scripts from the informal test case descriptions. Our results show that generative AI can produce correct or near-correct test scripts in a reasonable number of cases. We also analyzed the effects of prompt design, choice of generative AI model, and context accuracy on the effectiveness of our solution and reflected on our results.
@inproceedings{paperbot9097,
title = {Can Generative AI Produce Test Cases? An Experience from the Automotive Domain},
author = {Stephen Wynn-Williams and Ryan Tyrrell and Vera Pantelic and Mark Lawford and Claudio Menghi and Phaneendra Nalla and Hassan Artail},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3728568}
}