一句话：通过分类 CONTRIBUTING 文件并按入职障碍模型评估，发现多数项目未覆盖新人关键障碍。
开源项目常通过 CONTRIBUTING 文件指导新人，但这些文件缺乏标准结构、篇幅不一且可能遗漏新人常见障碍。本文提出自动解析 CONTRIBUTING 文件并评估其覆盖入职障碍的方法：人工按文献中的入职障碍模型标注样本，训练机器学习分类器预测段落类别，并用开发者调查检验预测合理性。分类器精确率 0.655、召回率 0.662，75% 的预测被开发者认为合理。分析发现 CONTRIBUTING 文件通常未覆盖新人障碍：52% 项目至少遗漏 6 类障碍中的 3 类，84% 至少遗漏 2 类；选择任务和与社区交流这两个高频障碍在超过 75% 项目中被忽视。作者将分类器开放为在线服务，可帮助社区维护者发现项目生态中缺失信息，也帮助新人理解 CONTRIBUTING 文件的预期。
Effectively onboarding newcomers is essential for the success of open source projects. These projects often provide onboarding guidelines in their ’CONTRIBUTING’ files (e.g., CONTRIBUTING.md on GitHub). These files explain, for example, how to find open tasks, implement solutions, and submit code for review. However, these files often do not follow a standard structure, can be too large, and miss barriers commonly found by newcomers. In this paper, we propose an automated approach to parse these CONTRIBUTING files and assess how they address onboarding barriers. We manually classified a sample of files according to a model of onboarding barriers from the literature, trained a machine learning classifier that automatically predicts the categories of each paragraph (precision: 0.655, recall: 0.662), and surveyed developers to investigate their perspective of the predictions’ adequacy (75% of the predictions were considered adequate). We found that CONTRIBUTING files typically do not cover the barriers newcomers face (52% of the analyzed projects missed at least 3 out of the 6 barriers faced by newcomers; 84% missed at least 2). Our analysis also revealed that information about choosing a task and talking with the community, two of the most recurrent barriers newcomers face, are neglected in more than 75% of the projects. We made available our classifier as an online service that analyzes the content of a given CONTRIBUTING file. Our approach may help community builders identify missing information in the project ecosystem they maintain and newcomers can understand what to expect in CONTRIBUTING files.
@inproceedings{FronchettiSWTGS23,
title = {Do CONTRIBUTING Files Provide Information about OSS Newcomers’ Onboarding Barriers?},
author = {Felipe Fronchetti and David C. Shepherd and Igor Wiese and Christoph Treude and Marco Aurélio Gerosa and Igor Steinmacher},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3616288}
}