paperbot · PL 论文追踪

全部论文 待读 收藏

P2 On the use of mutation analysis for evaluating student test suite quality

论使用变异分析评估学生测试套件质量
ISSTA Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis2022匹配度 1/10 · 论文关注变异分析在学生测试套件质量评估中的应用,属于软件测试与计算机教育方向;用户研究方向集中在 LLM/Agent 与静态分析、形式化验证、浮点程序分析和贝叶斯程序分析的交叉,两者几乎没有直接关联。引用 6 inbox
James Perretta, Andrew DeOrio, Arjun Guha, Jonathan Bell
crossref: 2026-09-12

AI 速览

一句话:研究变异分析自动生成的变异体能否替代人工植入缺陷与“全配对”评分,用于评估学生测试套件的质量。

在计算机科学课程中,评估学生编写的测试套件通常有两种做法:对照教师手工植入的缺陷,或与所有其他学生的实现进行“全配对”比较。前者耗时且易出错,后者在公平准确应用上需要大量人工与计算开销。变异分析能够自动在实现中植入潜在缺陷,是一种可能的替代方案。已有文献表明,在大规模开源软件项目中变异体可以替代真实缺陷,但变异体是否代表学生常犯的错误尚不清楚。本文考察变异体能否有效替代学生代码中的缺陷,以及变异体的检测是否与手工植入缺陷检测、错误学生实现检测相关;若成立,教师便可使用开源变异分析工具生成的变异体来评估学生测试套件。

问题
课程中评估学生测试套件依赖人工植入缺陷或耗力的“全配对”评分,需要寻找自动化且可靠的替代评估手段。
方法
研究使用变异分析自动生成变异体,并考察其与学生代码真实缺陷的对应关系,以及与手工植入缺陷检测、错误学生实现检测之间的相关性。
评估
通过实证比较变异体检测结果与手工植入缺陷检测、错误学生实现检测之间的相关性来评估变异分析是否可替代传统评分方式。
原文摘要(Abstract)

A common practice in computer science courses is to evaluate student-written test suites against either a set of manually-seeded faults (handwritten by an instructor) or against all other student-written implementations (“all-pairs” grading). However, manually seeding faults is a time consuming and potentially error-prone process, and the all-pairs approach requires significant manual and computational effort to apply fairly and accurately. Mutation analysis, which automatically seeds potential faults in an implementation, is a possible alternative to these test suite evaluation approaches. Although there is evidence in the literature that mutants are a valid substitute for real faults in large open-source software projects, it is unclear whether mutants are representative of the kinds of faults that students make. If mutants are a valid substitute for faults found in student-written code, and if mutant detection is correlated with manually-seeded fault detection and faulty student implementation detection, then instructors can instead evaluate student test suites using mutants generated by open-source mutation analysis tools.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot7427,
  title = {On the use of mutation analysis for evaluating student test suite quality},
  author = {James Perretta and Andrew DeOrio and Arjun Guha and Jonathan Bell},
  booktitle = {ISSTA},
  year = {2022},
  doi = {10.1145/3533767.3534217}
}