一句话：复现十年前ICSE调查，发现实证软件工程界对效度权衡与复现仍存分歧，呼吁更精细评审标准。
十年前的ICSE'15调查显示，软件工程研究界对实证研究中的内部效度与外部效度权衡存在分歧与困惑。本文复现该调查，分析当前领域关键研究者的790份开放式回答，考察十年间观点是否发生变化。结果表明，尽管研究者对开展实证研究的复杂性有了更高意识，但在什么构成好研究以及评审过程应如何体现这一点上，仍远未形成足够共识。参与者对内部效度、外部效度与生态效度的平衡意见不一；虽然普遍认同复现研究有必要，但何时、如何以及复现什么仍不清晰。作者据此呼吁采用更精细的评审流程，为不同方法制定清晰的实证标准，并促进关于哪些工作值得复现的诚实讨论。
Ten years ago, a survey (ICSE’15) revealed an unsatisfactory picture among key players of the software engineering research community: There was dissent and confusion on how to approach empirical research. Specifically, researchers were divided on the tradeoff between internal and external validity, holding strong and often opposing opinions, which cannot be a basis for stringent scientific progress. Clearly, the field has progressed over the last decade, but did the views progress, as well, or is the community still divided and confused? Our study addresses these questions by replicating the original survey among current key players of the field. Analyzing 790 open answers learning how perspectives have changed over the past ten years, we found that, despite increased awareness of the intricacies of conducting empirical studies, not nearly enough has changed to address the wide range of opinions on what a good study is and how that can be reflected in the review process. Specifically, participants disagree on balancing internal, external, and ecological validity, and while there is consensus on the need for replication studies, the specifics of when, how, and what to replicate remain unclear. Our results suggest the need for a more sophisticated review process, incorporating clear empirical standards for various methods and fostering honest discussions on what is worth replicating.
@inproceedings{paperbot11527,
title = {Views on Internal and External Validity in Empirical Software Engineering: 10 Years Later and Beyond},
author = {Alina Mailach and Janet Siegmund and Sven Apel and Norbert Siegmund},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3764520}
}