一句话:两阶段实证研究表明认知反思能力与人工识别代码缺陷相关,但与单元测试准确性无关。
验证代码是否满足规范需要批判性思维与分析,但认知偏差可能影响软件工程师的验证与测试效果。本文通过两阶段探索性研究,考察软件工程本科生(n=140)在认知反思测试(CRT)、人工验证函数实现以及单元测试套件准确性上的表现。第一阶段未发现CRT与单元测试准确性之间存在关系;但CRT得分越高的学生,在人工检查时越可能拒绝有缺陷的实现(p<0.0001,95% CI 1.56-4.50)。第二阶段使用不同版本的CRT和不同的单元测试作业复现了该结果,发现CRT与人工识别缺陷之间存在正相关(ρ=0.478,p<0.01)。作者结论:认知反思与软件工程学生识别缺陷的能力相关,但与确认无缺陷算法无关,也与单元测试准确性无关。
Verifying whether code meets its specifications requires critical thinking and analysis. However, cognitive biases may influence how well software engineers verify and test code. In this paper, I explore cognitive reflection and its association with accurately verifying code via both manual inspection and unit testing. In a two-phase exploratory study of Software Engineering undergraduate students (n=140), I examined their performance on Cognitive Reflection Tests (CRT), manual verification of function implementations, and the accuracy of unit test suites. The first phase found no relationship between CRT and unit test accuracy. However, the higher a student’s CRT score, the more likely they were to reject a defective implementation when inspecting it manually (p<0.0001, 95% CI 1.56-4.50). The second phase replicated the outcomes on an alternate version of the CRT and on a different unit test assignment, revealing a positive correlation between CRT and recognizing defects manually ($\rho$=0.478, p<0.01). I conclude that cognitive reflection is associated with software engineering students’ aptitude at identifying defects, but is not associated with their affirmation of algorithms without defects, nor with the accuracy of their unit tests.