一句话:通过两阶段混合方法研究学生在软件测试中使用LLM的困难与交互模式,并提出阶段感知提示支架。
随着LLM进入软件测试实践,学生日益用其完成测试任务,但其对测试行为与学习的影响尚不清楚。本文开展两阶段混合方法探索性研究,考察软件测试教育中的人-LLM协作。第一阶段分析15名学生的课堂制品与交互记录,并结合全国软件测试竞赛调查(337份有效回复),识别出上下文缺失、约束不足、一次性提示及策略迭代有限等困难;自动测试脚本生成最具异质性且高投入。第二阶段据此设计轻量、阶段感知的提示支架用于测试脚本生成,并报告学生执行相关信息表述上的描述性变化。研究刻画任务相关困难与交互模式,为以学习为导向地将LLM整合进软件测试教育提供实证基础。
Large Language Models (LLMs) are rapidly entering software engineering practice, including software testing. As students increasingly use LLMs in testing tasks, software testing education must adapt. Yet we still know little about how such use shapes students' testing behaviors, judgment, and learning. Systematic investigation is therefore needed. This paper reports a mixed-methods, two-phase exploratory study of human-LLM collaboration in software testing education. In Phase I, we analyze classroom artifacts and interaction records from 15 students together with a national software testing competition survey (337 valid responses) to identify recurring prompt-related difficulties across testing tasks. We find missing context, insufficient constraints, rigid one-shot prompting, and limited strategy-driven iteration. Automated test script generation emerges as the most heterogeneous and effort-intensive context. In Phase II, we translate these findings into a lightweight, stage-aware prompt scaffold for test script generation and report descriptive shifts in students' articulation of execution-relevant information. From a learning-behavior perspective, this paper characterizes task-dependent difficulties and interaction patterns in students' use of LLMs for software testing. It distinguishes relatively stable interaction contexts from more heterogeneous ones and shows how observed difficulties can be translated into classroom support. Overall, the findings provide an empirical foundation for learning-oriented integration of LLMs into software testing education.
DOI 原文 · arXiv · PDF(开放获取) · DBLP
@inproceedings{abs-2603-26329,
title = {Large Language Models for Software Testing Education: an Experience Report},
author = {Peng Yang and Yunfeng Zhu and Chao Chang and Shengcheng Yu and Zhenyu Chen and Yong Tang},
booktitle = {FSE},
year = {2026},
doi = {10.1145/3803437.3805782}
}