一句话：开发端到端工具链，用扩展 KLEE 对 LC-3 学生汇编程序做等价性检查、静态检查与反向调试反馈。
本文为 LC-3 汇编教学构建端到端反馈工具链，解决学生难以及时发现代码问题与行为差异的痛点。系统核心是扩展 KLEE 得到的 KLC3，可将学生程序与标准答案协同执行，进行问题识别与等价性检查。学生通过 VSCode 扩展获得静态正确性与风格检查；提交到 Git 后自动触发 KLC3，以可读报告、测试用例和脚本返回差异。系统还提供队列查看、LC-3 反向调试和基于 Emscripten 的浏览器界面，并为每位学生维护回归测试，要求先通过历史测试再复评，避免盲目试错。2020 年秋季在 100 余名学生的课程中部署，学生编写约 700 行汇编，匿名反馈一致积极。
We developed a set of tools designed to provide rapid feedback to students as they learn to write programs in assembly language (LC-3, a RISC-like educational instruction set architecture). At the heart of the system is an extended version of KLEE, KLC3, that enables us to both identify issues and perform equivalence checking between student code and a gold (correct) version of each assignment. Feedback begins when students edit their code using a VSCode extension that leverages static analysis to perform a variety of correctness and style checks, encouraging students to improve their code quality. Each time a student commits code to their Git repository, our system triggers. Using KLC3 (KLEE), the student code is executed along with the gold version, and issues and behavioral differences are delivered back to the student through their Git repository as a human-readable report, test cases, and scripts. A queueing system allows students to monitor progress, but responses are generally available within minutes. We also extended the LC-3 simulation tools to support reverse debugging, making the process of finding complex bugs much more tractable for students, and used Emscripten to develop a browser-based interface for use in testing and debugging. Finally, our system maintains an individual regression test suite for each student and requires a submission to pass all previous tests before re-evaluation in KLC3, thus avoiding encouraging programming-by-guesswork. We deployed the system to provide feedback for the assembly programming assignments in a class of over 100 students in Fall 2020. Students wrote a median of around 700 lines of assembly for these assignments, making heavy use of our tools to understand and eliminate their bugs. Anonymous student feedback on the tools was uniformly positive. Since that semester, we have continued to refine and expand our tools’ analysis capabilities and performance, and plan to deploy the system again in the near future (the class is offered every Fall).
DOI 原文 ·
@inproceedings{paperbot6635,
title = {End-to-End Automation of Feedback on Student Assembly Programs},
author = {Zikai Liu and Tingkai Liu and Qi Li and Wenqing Luo and Steven S. Lumetta},
booktitle = {ASE},
year = {2021},
doi = {10.1109/ase51524.2021.9678837}
}