paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Large Language Models for Software Testing Education: an Experience Report

大语言模型用于软件测试教育:一项经验报告
FSE Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering2026匹配度 2/10 · 该论文关注软件测试教育中的人-LLM交互、提示困难与教学支架,属于教育实证和提示工程方向;与用户核心关注的静态分析、形式化验证、浮点程序分析、贝叶斯程序分析及其与LLM的交叉均无直接关系。引用 0 inbox
Peng Yang, Yunfeng Zhu, Chao Chang, Shengcheng Yu, Zhenyu Chen, Yong Tang
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:通过两阶段混合方法研究学生在软件测试中使用LLM的困难与交互模式,并提出阶段感知提示支架。

随着LLM进入软件测试实践,学生日益用其完成测试任务,但其对测试行为与学习的影响尚不清楚。本文开展两阶段混合方法探索性研究,考察软件测试教育中的人-LLM协作。第一阶段分析15名学生的课堂制品与交互记录,并结合全国软件测试竞赛调查(337份有效回复),识别出上下文缺失、约束不足、一次性提示及策略迭代有限等困难;自动测试脚本生成最具异质性且高投入。第二阶段据此设计轻量、阶段感知的提示支架用于测试脚本生成,并报告学生执行相关信息表述上的描述性变化。研究刻画任务相关困难与交互模式,为以学习为导向地将LLM整合进软件测试教育提供实证基础。

问题
软件测试教育中,学生使用LLM的方式及其对测试行为、判断和学习的影响尚缺乏系统理解。
方法
采用两阶段混合方法:分析15名学生的课堂制品与交互记录及337份竞赛调查,识别提示困难;再据此设计阶段感知提示支架并观察变化。
评估
通过课堂制品、交互记录与全国调查进行混合方法分析,并以描述性方式报告提示支架前后的变化。
原文摘要(Abstract)

Large Language Models (LLMs) are rapidly entering software engineering practice, including software testing. As students increasingly use LLMs in testing tasks, software testing education must adapt. Yet we still know little about how such use shapes students' testing behaviors, judgment, and learning. Systematic investigation is therefore needed. This paper reports a mixed-methods, two-phase exploratory study of human-LLM collaboration in software testing education. In Phase I, we analyze classroom artifacts and interaction records from 15 students together with a national software testing competition survey (337 valid responses) to identify recurring prompt-related difficulties across testing tasks. We find missing context, insufficient constraints, rigid one-shot prompting, and limited strategy-driven iteration. Automated test script generation emerges as the most heterogeneous and effort-intensive context. In Phase II, we translate these findings into a lightweight, stage-aware prompt scaffold for test script generation and report descriptive shifts in students' articulation of execution-relevant information. From a learning-behavior perspective, this paper characterizes task-dependent difficulties and interaction patterns in students' use of LLMs for software testing. It distinguishes relatively stable interaction contexts from more heterogeneous ones and shows how observed difficulties can be translated into classroom support. Overall, the findings provide an empirical foundation for learning-oriented integration of LLMs into software testing education.

链接与引用

DOI 原文 · arXiv · PDF(开放获取) · DBLP

BibTeX
@inproceedings{abs-2603-26329,
  title = {Large Language Models for Software Testing Education: an Experience Report},
  author = {Peng Yang and Yunfeng Zhu and Chao Chang and Shengcheng Yu and Zhenyu Chen and Yong Tang},
  booktitle = {FSE},
  year = {2026},
  doi = {10.1145/3803437.3805782}
}