一句话：提出 Grasa，通过聚类错误提交并生成最小额外测试，增强教师测试套件以检出错误提交。
CS1 入门编程课程选课人数激增，单个问题可产生数千份提交，教师需要及时准确评分。常见做法是构建教师自定义测试套件，在随机或手工输入上比较学生提交与参考解，但该套件往往不够充分，导致错误提交被误判为正确。为此，论文提出 Grasa（基于评分的测试套件增强）方法，包含两项技术：首先通过近似提交之间的行为等价来检测并聚类错误提交；然后生成一组最小的额外测试，以帮助检出这些错误提交，从而增强现有教师测试套件。作者在三个编程问题的 CS1 学生提交数据集上评估 Grasa，初步结果表明该方法能有效识别错误学生提交，并以最小方式增强教师测试套件。
Enrollment in introductory programming (CS1) courses continues to surge and hundreds of CS1 students can produce thousands of submissions for a single problem, all requiring timely and accurate grading. One way that instructors can efficiently grade is to construct a custom instructor test suite that compares a student submission to a reference solution over randomly generated or hand-crafted inputs. However, such test suite is often insufficient, causing incorrect submissions to be marked as correct. To address this issue, we propose the Grasa (GRAding-based test Suite Augmentation) approach consisting of two techniques. Grasa first detects and clusters incorrect submissions by approximating their behavioral equivalence to each other. To augment the existing instructor test suite, Grasa generates a minimal set of additional tests that help detect the incorrect submissions. We evaluate our Grasa approach on a dataset of CS1 student submissions for three programming problems. Our preliminary results show that Grasa can effectively identify incorrect student submissions and minimally augment the instructor test suite.
DOI 原文 ·
@inproceedings{paperbot5238,
title = {Grading-Based Test Suite Augmentation},
author = {Jonathan Osei-Owusu and Angello Astorga and Liia Butler and Tao Xie and Geoffrey Challen},
booktitle = {ASE},
year = {2019},
doi = {10.1109/ase.2019.00030}
}