paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Cognitive Reflection in Software Verification and Testing

软件验证与测试中的认知反思
ICSE 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET)2023匹配度 1/10 · 论文研究人类认知反思能力与人工代码缺陷识别、单元测试准确性之间的实证关系,属于软件工程教育/认知心理学方向;与用户关注的静态分析、LLM/Agent 程序分析、浮点程序分析及贝叶斯程序分析没有直接交集。引用 0 inbox
Kevin Buffardi
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:两阶段实证研究表明认知反思能力与人工识别代码缺陷相关,但与单元测试准确性无关。

验证代码是否满足规范需要批判性思维与分析,但认知偏差可能影响软件工程师的验证与测试效果。本文通过两阶段探索性研究,考察软件工程本科生(n=140)在认知反思测试(CRT)、人工验证函数实现以及单元测试套件准确性上的表现。第一阶段未发现CRT与单元测试准确性之间存在关系;但CRT得分越高的学生,在人工检查时越可能拒绝有缺陷的实现(p<0.0001,95% CI 1.56-4.50)。第二阶段使用不同版本的CRT和不同的单元测试作业复现了该结果,发现CRT与人工识别缺陷之间存在正相关(ρ=0.478,p<0.01)。作者结论:认知反思与软件工程学生识别缺陷的能力相关,但与确认无缺陷算法无关,也与单元测试准确性无关。

问题
验证代码是否符合规范需要批判性思维,但认知偏差可能影响软件工程师验证和测试代码的效果。
方法
对140名软件工程本科生开展两阶段探索性研究,测量其认知反思测试(CRT)成绩、人工验证函数实现的准确性和单元测试套件准确性。
评估
第一阶段分析CRT与单元测试准确性及人工验证表现的关系;第二阶段用不同版本的CRT和不同的单元测试作业进行复现,均通过统计检验评估相关性。
原文摘要(Abstract)

Verifying whether code meets its specifications requires critical thinking and analysis. However, cognitive biases may influence how well software engineers verify and test code. In this paper, I explore cognitive reflection and its association with accurately verifying code via both manual inspection and unit testing. In a two-phase exploratory study of Software Engineering undergraduate students (n=140), I examined their performance on Cognitive Reflection Tests (CRT), manual verification of function implementations, and the accuracy of unit test suites. The first phase found no relationship between CRT and unit test accuracy. However, the higher a student’s CRT score, the more likely they were to reject a defective implementation when inspecting it manually (p<0.0001, 95% CI 1.56-4.50). The second phase replicated the outcomes on an alternate version of the CRT and on a different unit test assignment, revealing a positive correlation between CRT and recognizing defects manually ($\rho$=0.478, p<0.01). I conclude that cognitive reflection is associated with software engineering students’ aptitude at identifying defects, but is not associated with their affirmation of algorithms without defects, nor with the accuracy of their unit tests.

链接与引用

DOI 原文 · DBLP

BibTeX
@inproceedings{Buffardi23,
  title = {Cognitive Reflection in Software Verification and Testing},
  author = {Kevin Buffardi},
  booktitle = {ICSE},
  year = {2023},
  doi = {10.1109/icse-seet58685.2023.00006}
}