一句话：回顾十年软件工程会议工件评估的演变，审视其目标达成度与开销合理性。
大多数软件工程研究都会产出原型、概念验证或测量装置，它们与数据一起构成研究工件，并在会议中接受工件评估（AE）。自 ESEC/FSE 2011 在软件工程社区启动 AE 以来，其目标与流程不断演变，当前期望与可复现研究结果和可复用工具有紧密关联。然而，至今仍缺乏证据表明通过 AE 的工件真正达到了这些高期望，即 AE 流程在多大程度上促进了其目标，以及其带来的额外开销是否合理。本文对十年 AE 进行回顾性研究，系统考察其演变并分析上述问题，但摘要未披露具体结果数据。
Most software engineering research involves the development of a prototype, a proof of concept, or a measurement apparatus. Together with the data collected in the research process, they are collectively referred to as research artifacts and are subject to artifact evaluation (AE) at scientific conferences. Since its initiation in the SE community at ESEC/FSE 2011, both the goals and the process of AE have evolved and today expectations towards AE are strongly linked with reproducible research results and reusable tools that other researchers can build their work on. However, to date little evidence has been provided that artifacts which have passed AE actually live up to these high expectations, i.e., to which degree AE processes contribute to AE's goals and whether the overhead they impose is justified.
@inproceedings{paperbot6963,
title = {A retrospective study of one decade of artifact evaluations},
author = {Stefan Winter and Christopher S. Timperley and Ben Hermann and Jürgen Cito and Jonathan Bell and Michael Hilton and Dirk Beyer},
booktitle = {FSE},
year = {2022},
doi = {10.1145/3540250.3549172}
}