paperbot · PL 论文追踪

全部论文 待读 收藏

P2 The Impact of Documentation on Test Engagement in Pull Requests in OSS

开源软件中测试文档对 Pull Request 测试参与度的影响
FSE Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering2026匹配度 1/10 · 该论文属于经验软件工程与开发者行为研究,关注测试文档与 OSS PR 测试参与度的相关性,未涉及静态分析、LLM/Agent、浮点程序分析或贝叶斯程序分析等用户核心研究方向。引用 0 inbox
Teal Amore, Nathan Berman, Siyuan Jiang
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:基于 160 个 OSS 仓库,提出测试参与比 TER,发现测试文档完备性与测试参与度呈弱至中度正相关。

自动测试对维护开源软件质量至关重要,但如何激励贡献者为代码变更添加测试仍是难题。现有干预(如覆盖率指标、评审反馈)多在 Pull Request 打开后才被动实施。本文研究测试相关文档能否作为主动措施促进测试行为,并提出 Test Engagement Ratio(TER)衡量测试频率。作者基于 160 个 OSS 仓库的数据,分析文档完备性与 TER 的关系。结果显示二者存在弱但统计显著的正相关(ρ=0.36, p<0.001),在 PR 活动更活跃的仓库中增强为中度相关(ρ=0.44)。How to Run Tests 与 How to Write Tests 两类文档与测试参与度相关性最强。此外,TER 与 Test Code Ratio 呈中度相关(ρ=0.52, p<0.001),初步支持其有效性。研究表明测试文档可能与更高的测试参与度相关,未来将探索因果性、细粒度文档质量和跨仓库曝光效应。

问题
如何主动激励开源贡献者在 Pull Request 中编写测试,而非仅依赖代码覆盖率或评审反馈等事后干预。
方法
提出 Test Engagement Ratio(TER),基于 160 个 OSS 仓库分析测试文档完备性与 TER 的相关性,并检验 TER 与 Test Code Ratio 的关系。
评估
对 160 个仓库进行相关性分析;发现文档完备性与 TER 弱正相关,在高 PR 活动仓库中为中度相关,TER 与 Test Code Ratio 中度相关。
原文摘要(Abstract)

Automated testing is crucial for maintaining open-source software quality. However, motivating contributors to include tests for code changes remains a challenge. While existing interventions, such as code coverage metrics and reviewer feedback, are often reactive and applied only after a pull request is opened, this study investigates whether documentation on testing can serve as a proactive measure to encourage testing behavior. In this work, we investigate the relationship between documentation on testing and contributor testing behavior. We introduce the Test Engagement Ratio (TER) to help understand testing frequency. Using data from 160 OSS repositories, we analyze the relationship between documentation comprehensiveness and TER. Our results show a weak but statistically significant positive correlation (ρ = 0.36, p < 0.001), which strengthens to a moderate relationship (ρ = 0.44) in repositories with higher pull request activity. Documentation categories such as How to Run Tests and How to Write Tests show the strongest correlation with testing engagement. Furthermore, TER is found to be moderately correlated (ρ = 0.52, p < 0.001) with Test Code Ratio, providing preliminary evidence of its validity. Our findings suggest that documentation on testing may be associated with increased testing engagement. Future work will explore causality, documentation quality at a granular level, and cross-repository exposure effects.

链接与引用

DOI 原文 · arXiv · PDF(开放获取) · DBLP

BibTeX
@inproceedings{abs-2604-23048,
  title = {The Impact of Documentation on Test Engagement in Pull Requests in OSS},
  author = {Teal Amore and Nathan Berman and Siyuan Jiang},
  booktitle = {FSE},
  year = {2026},
  doi = {10.1145/3803437.3805555}
}