一句话：揭示实证软件工程中同一工具既作测量又作干预时产生的处理敏感因素结构偏差，并给出缓解建议。
在实证软件工程中，研究者常将程序分析工具用于实验：既测量软件属性（因变量），又作为干预（自变量）的一部分。当同一工具同时承担测量仪器与干预角色时，可能产生“处理敏感的因素结构偏差”，即接受干预的参与者对所评估构念的体验与其他参与者不同，从而使结果偏向某一组。本文对这一问题进行反思性分析，刻画其作为构念效度威胁的机制，并讨论缓解策略。作者建议：尽量使用与处理无关的测量工具；若不可行，应使测量工具与处理之间的关联尽可能小；此外，选择测量工具的合适版本和/或配置也有助于减轻该偏差。本文不开展新的实证基准，而是为实验或队列研究提供方法学建议。
Researchers in Software Engineering (SE) frequently use program analysis tools (or simply tools) as part of their experiments. These tools can be used to measure software attributes (response/dependent variables) or as part of an intervention (treatment/independent variable). But what happens when researchers use the same tool both as a measurement instrument and as part of an intervention? They might be dealing with a threat to construct validity: the treatment-sensitive factorial structure bias. Accordingly, the participants receiving the intervention might experience what is being evaluated in the experiment differently from the other participants. This means that the results of the study might favor one group of participants over others based on their exposure to the intervention (which involved the tool used to make measurements). In this paper, we present a reflection-based analysis of the treatment-sensitive factorial structure bias. We share suggestions on how to mitigate this threat in SE studies (e.g., experiments or cohort studies). We advise researchers to use measurement instruments not associated with the treatment. In case this is not possible, the association between the measurement instrument and the treatment should be as small as possible. Also, choosing a proper version and/or configuration of the measurement instrument might mitigate the treatment-sensitive factorial structure bias.
@inproceedings{paperbot8956,
title = {Missing Threats: Dealing with the Treatment-sensitive Factorial Structure Bias in Empirical Software Engineering},
author = {Sabato Nocera and Giuseppe Scanniello},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3728527}
}