一句话：通过实证分析 Travis CI 构建数据，揭示构建结果存在噪声且构建间异质，挑战将构建视为干净且等价的常见假设。
自动化构建为开发团队提供代码变更反馈：通过表示编译和测试成功，失败表示需要关注的问题。论文指出，若不对构建结果数据进行深入分析，实践者和研究者容易做出两个关键假设：一是构建结果无噪声，但实际上通过的构建可能包含失败或被跳过的 job，这些 job 被主动或被动忽略；二是所有构建等价，但构建在 job 数量和配置上存在差异。围绕 Travis CI 历史构建数据，论文开展实证研究，揭示构建结果中的噪声与构建间的异质性，说明直接使用构建通过/失败标签可能掩盖这些复杂情况，并为后续研究和工具实践提供警示。
Automated builds, which may pass or fail, provide feedback to a development team about changes to the codebase. A passing build indicates that the change compiles cleanly and tests (continue to) pass. A failing (a.k.a., broken) build indicates that there are issues that require attention. Without a closer analysis of the nature of build outcome data, practitioners and researchers are likely to make two critical assumptions: (1) build results are not noisy; however, passing builds may contain failing or skipped jobs that are actively or passively ignored; and (2) builds are equal; however, builds vary in terms of the number of jobs and configurations.
DOI 原文 ·
@inproceedings{paperbot4718,
title = {Noise and heterogeneity in historical build data: an empirical study of Travis CI},
author = {Keheliya Gallaba and Christian Macho and Martin Pinzger and Shane McIntosh},
booktitle = {ASE},
year = {2018},
doi = {10.1145/3238147.3238171}
}