一句话：提出 AutoVCS 算法与工具，自动汇总团队 GitHub 仓库中的个人贡献，提升助教评分一致性与反馈质量。
团队式学习在软件工程课程中广泛使用，但评估个人贡献颇具挑战：在 CS1.5 入门课程中，助教需手动评定学生每周实验的团队贡献，容易不一致且反馈有限。本文提出一种新算法并实现工具 AutoVCS，自动从团队 GitHub 仓库生成个人贡献摘要供助教评分时参考。作者在 NC State University CS1.5 课程的 Java 实验上进行控制实验，从评分速度、评分一致性和助教满意度三个维度评估。结果表明，自动化摘要帮助助教评分更一致，并能为学生提供更可操作的反馈；虽然评分速度没有加快，但助教强烈偏好使用摘要辅助评分。最后，论文为未来提升学生软件工程团队贡献评分一致性提出建议。
Teaming is a key aspect of most professional software engineering positions, and consequently, team-based learning (TBL) features heavily in many undergraduate computer science (CS) and software engineering programs. However, while TBL offers many pedagogical benefits, it is not without challenges. One such challenge is assessment, as the course teaching staff must be able to accurately identify individual students’ contributions to both encourage and reward participation. In this paper, we study improvements to grading practises in the context of a CS1.5 introductory software engineering course, where assessing individual students’ contributions to weekly lab assignments is done manually by teaching assistants (TAs). We explore the impact of presenting TAs with automated summaries of individual student contributions to their team’s GitHub repository. To do so, we propose a novel algorithm, and implement a tool based off of it, AutoVCS. We measure the impact on grading metrics in terms of grading speed, grading consistency, and TA satisfaction. We evaluate our algorithm, as implemented in AutoVCS, in a controlled experimental study on Java-based lab assignments from a recent offering of NC State University’s CS1.5 course. We find our automated summaries help TAs grade more consistently and provides students with more actionable feedback. Although TAs grade no faster using automated summaries, they nonetheless strongly prefer grading with the support of them than without. We conclude with recommendations for future work to explore improving consistency in contribution grading for student software engineering teams.
@inproceedings{PreslerMarshallHS23,
title = {Improving Grading Outcomes in Software Engineering Projects Through Automated Contributions Summaries},
author = {Kai Presler-Marshall and Sarah Heckman and Kathryn T. Stolee},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse-seet58685.2023.00030}
}