paperbot · PL 论文追踪

全部论文 待读 收藏

P2 The Impact of Generative AI on Test & Evaluation: Challenges and Opportunities

生成式人工智能对测试与评估的影响:挑战与机遇
FSE Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering2025匹配度 3/10 · 论文是泛论生成式AI对测试与评估/V&V影响的立场文章,涉及AI非确定性验证挑战,但与用户核心关注的静态分析、形式化验证、浮点程序分析、贝叶斯程序分析及Agentic程序分析无直接交集,且偏宏观讨论、缺乏具体方法与benchmark。引用 6 inbox
Laura Freeman, John Robert, Heather Wojton
crossref: 2026-09-12

AI 速览

一句话:立场论文分析生成式AI如何改变测试与评估,并讨论其非确定性带来的验证挑战与机遇。

生成式人工智能正在改变软件开发流程中的测试与评估(T&E):从自动设计测试用例到在DevOps流水线中实现持续测试,AI驱动工具提升了软件测试的效率、准确性和速度。与此同时,将AI组件集成到依赖软件的系统后,验证与确认(V&V)面临新挑战,传统T&E方法必须应对AI偏见、幻觉输出以及非确定性行为验证的复杂性。这篇立场论文考察现有T&E方法应如何演进以适应AI的随机性,并反向分析GenAI如何在整个软件开发生命周期中改变T&E实践。

问题
生成式AI进入软件生命周期后,传统测试与评估方法难以应对AI偏见、幻觉输出和非确定性行为,验证与确认面临新挑战。
方法
作为立场论文,作者梳理GenAI在测试用例设计、DevOps持续测试等T&E环节的应用,并分析T&E方法为适配AI随机性所需的演进。
评估
论文通过立场分析与问题讨论提出挑战和机遇,未开展新的实证研究或工具评估。
原文摘要(Abstract)

Generative Artificial Intelligence (GenAI) is transforming software development processes, including test and evaluation (T&E). From automating test case design to enabling continuous testing in DevOps pipelines, AI-driven tools enhance the efficiency, accuracy, and speed of software testing. At the same time, the integration of AI components into software-reliant systems introduces new challenges for verification and validation (V&V). Traditional T&E methodologies must evolve to address issues such as AI bias, hallucinated outputs, and the complexity of validating non-deterministic behaviors. This position paper examines how existing T&E methods must evolve to account for AI's stochastic nature, and conversely how GenAI is transforming T&E practices across the software development lifecycle (SDLC).

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot9133,
  title = {The Impact of Generative AI on Test & Evaluation: Challenges and Opportunities},
  author = {Laura Freeman and John Robert and Heather Wojton},
  booktitle = {FSE},
  year = {2025},
  doi = {10.1145/3696630.3728723}
}