paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Artifact Evaluations for Stronger Research Results

面向更可靠研究结果的工件评估
FSE Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering2025匹配度 2/10 · 这篇论文讨论软件工程社区的工件评估与可复现性,属于研究方法/社区机制层面,而非 PL 与 AI 交叉、静态分析、浮点程序分析或贝叶斯程序分析等核心方向。虽然可复现性和工件评审与工程落地、benchmark 可信度有一定间接关联,但并非用户当前追的问题。引用 2 inbox
Dirk Beyer, Stefan Winter
crossref: 2026-09-12

AI 速览

一句话:回顾软件工程社区工件评估的兴起与运作,说明其如何提升研究结果的可信度。

过去数十年,多个科学社区出现可复现性危机:独立复现已发表结果失败,使结果有效性受到质疑。软件工程社区较早应对该问题,于 ESEC/FSE 2011 首次引入工件评估(artifact evaluation),作者自愿提交支撑论文结果的工件,接受同行评审。此后工件评估在软件工程会议中迅速普及,多数会议都提供该环节,并形成规模庞大的工件评估委员会来处理大量提交。部分会议还将工件评估与论文录用挂钩,要求工件通过评估论文才可发表。本文讨论这一实践如何增强研究结果的可靠性。

问题
已发表研究结果难以独立复现,可复现性危机动摇其有效性。
方法
在软件工程会议中引入工件评估:作者自愿提交支撑结果的工件,由同行评审委员会审查。
评估
以 ESEC/FSE 2011 为起点,追踪工件评估在多数软件工程会议的普及、委员会规模与提交量,以及部分会议将工件通过评估作为论文录用条件。
原文摘要(Abstract)

Over the past decades, reports of reproducibility crises have surfaced in various scientific communities. Independent confirmations of published research results failed, casting doubt on the validity of these results. Even before the magnitude of the problem has become apparent in many domains, the software-engineering community introduced artifact evaluations, for the first time at ESEC/FSE 2011, in which research artifacts that support published results were voluntarily submitted for peer review. Since then, artifact evaluations have become immensely popular and are today being offered to authors at most software-engineering venues, where large artifact-evaluation committees handle large numbers of artifact submissions. At some venues, papers are accepted for publication only if their artifacts pass the artifact evaluation.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot8937,
  title = {Artifact Evaluations for Stronger Research Results},
  author = {Dirk Beyer and Stefan Winter},
  booktitle = {FSE},
  year = {2025},
  doi = {10.1145/3696630.3728623}
}