一句话:提出 FAIL 作业,用 STAMP 模型系统分析历史软件事故,以培养软件测试初学者的安全意识和软技能。
软件测试初学者往往难以体会测试不足的严重后果。本文提出 FAIL 作业,让学生系统调查 13 起历史软件事故,从 Ariane-5 火箭爆炸到 Knight Capital 交易故障,并采用基于系统理论的 STAMP 事故因果模型;学生还有机会采访 Therac-25 辐射过量事故的见证人。该作业在 96 名计算机科学毕业生中部署,获得正面评价。结果表明,这种轻量但系统化的失败调查能激励学生,使其对次优测试的后果更敏感;还揭示测试人员所需的防御性悲观、坚定立场等关键软技能。真实案例比人工案例更具可信度,留下的测试场景也令学生难忘。
Accidents tend to be traumatic events that one would rather forget than remember. Software testing novices at the Vrije Universiteit in Amsterdam, on the contrary, rewind the past and learn how to safeguard the future.In this paper we will present FAIL, a rather unconventional assignment that methodically investigate 13 historical software-related accidents, varying from the Ariane-5 rocket explosion to the Knight Capital trading glitch. Innovative is that software testing students use STAMP, a modern systems-theory-based accident causality model and have the possibility to interview a witness of the famous Therac-25 radiation overexposures.A recent deployment to 96 CS graduates received positive evaluations. We learned that even a lightweight, yet systematic investigation of failures (1) motivates students, by sensitizing them to the consequences of suboptimal testing, and (2) reveals key soft-skills testers need to prevent disasters, such as defensive pessimism and a strong backbone. Other, more subtle benefits of the proposed approach include (3) really-happened, instead of artificial case-studies that increase a teacher’s credibility, and (4) extraordinary test scenarios students will always remember.These results invite software engineering educators to include safety assessment elements in their curricula, and call on witnesses of software-related accidents to break the silence and share memories. Future work includes crafting a repository of heritage artifacts (narratives, videos, witness testimonies and physical replicas) to reproduce historical software-related accidents, and make it available to interested educators. Our hope is that motivated professionals will emerge, better prepared to engineer the safe software-intensive systems we all can rely on.
@inproceedings{SilvisCividjianH23,
title = {Speak, Memory! Analyzing Historical Accidents to Sensitize Software Testing Novices},
author = {Natalia Silvis-Cividjian and Fritz Hager},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse-seet58685.2023.00013}
}