一句话：通过随机实验评估 Alloy 形式规约学习中不同类型自动提示对表现、学习保持与情绪的影响。
本文研究自动提示在形式规约语言学习中的作用。作者以 Alloy 为对象，对 85 名本科生开展两阶段随机实验：第一阶段分对照组与三组接受不同提示学习需求规约，一周后第二阶段取消提示，并前后用情绪量表测量。结果显示，仅指出错误精确位置的提示改善了即时表现，但三类提示对学习保持均无显著影响；提示可用性显著影响情绪，移除后无剥夺效应。结论是在形式规约学习环境中提供精确定位错误的提示可改善体验且无明显弊端，未来需检验结合自我解释提示等其他反馈能否促进保持。
Background: Many programming environments include automated feedback in the form of hints to help novices learn autonomously. Some experimental studies investigated the impact of automated hints in the immediate performance and learning retention in that context. Automated feedback is also becoming a popular research topic in the context of formal specification languages, but so far no experimental studies have been conducted to assess its impact while learning such languages. Objective: We aim to investigate the impact of different types of automated hints while learning a formal specification language, not only in terms of immediate performance and learning retention, but also in the emotional response of the students. Method: We conducted a simple one-factor randomised experiment in 2 sessions involving 85 BSc students majoring in CSE. In the 1st session students were divided in 1 control group and 3 experimental groups, each receiving a different type of hint while learning to specify simple requirements with the Alloy formal specification language. To assess the impact of hints on learning retention, in the 2nd session, 1 week later, students had no hints while formalising requirements. Before and after each session the students answered a standard self-reporting emotional survey to assess their emotional response to the experiment. Results: Of the 3 types of hints considered, only those pointing to the precise location of an error had a positive impact on the immediate performance and none had significant impact in learning retention. Hint availability also causes a significant impact on the emotional response, but no significant emotional impact exists once hints are no longer available (i.e. no deprivation effects were detected). Conclusion: Although none of the evaluated hints had an impact on learning retention, learning a formal specification language with an environment that provides hints with precise error locations seems to contribute to a better overall experience without apparent drawbacks. Further studies are needed to investigate if other kind of feedback, namely hints combined with some sort of self-explanation prompts, can have a positive impact in learning retention.
@inproceedings{CunhaMCMS24,
title = {Assessing the impact of hints in learning formal specification},
author = {Alcino Cunha and Nuno Macedo and José Creissac Campos and Iara Margolis and Emanuel Sousa},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3639474.3640050}
}