一句话：通过挖掘 GitHub 仓库，比较不同程度采用 TDD 的项目，未发现可观测收益。
该研究针对测试驱动开发（TDD）能否降低成本、提升质量开展成本-收益分析。不同于实验室研究，作者在真实 GitHub 生态中按 TDD 严格程度对 2015 年归档仓库分类，构建 TDD 谱系，并将不同 TDD 子集与在测试文件等特征上匹配的等规模对照组进行比较。考察指标包括测试文件数、平均提交速度、缺陷相关提交数、issue 数、持续集成使用、pull request 数及作者提交分布。结果显示 Java TDD 项目相对稀少，且 TDD 类与非 TDD 项目在各指标上很少有显著差异，因此未发现采用 TDD 带来的可观测收益。
Testing is an integral part of the software development lifecycle, approached with varying degrees of rigor by different process models. Agile process models recommend Test Driven Development (TDD) as a key practice for reducing costs and improving code quality. The objective of this work is to perform a cost-benefit analysis of this practice. Previous work by Fucci et al. [2, 3] engaged in laboratory studies of developers actively engaged in test-driven development practices. Fucci et al. found little difference between test-first behaviour of TDD and test-later behaviour. To that end, we opted to conduct a study about TDD behaviours in the "wild" rather than in the laboratory. Thus we have conducted a comparative analysis of GitHub repositories that adopts TDD to a lesser or greater extent, in order to determine how TDD affects software development productivity and software quality. We classified GitHub repositories archived in 2015 in terms of how rigorously they practiced TDD, thus creating a TDD spectrum. We then matched and compared various subsets of these repositories on this TDD spectrum with control sets of equal size. The control sets were samples from all GitHub repositories that matched certain characteristics, and that contained at least one test file. We compared how the TDD sets differed from the control sets on the following characteristics: number of test files, average commit velocity, number of bug-referencing commits, number of issues recorded, usage of continuous integration, number of pull requests, and distribution of commits per author. We found that Java TDD projects were relatively rare. In addition, there were very few significant differences in any of the metrics we used to compare TDD-like and non-TDD projects; therefore, our results do not reveal any observable benefits from using TDD.
DOI 原文 ·
@inproceedings{paperbot4497,
title = {Analyzing the effects of test driven development in GitHub},
author = {Neil C. Borle and Meysam Feghhi and Eleni Stroulia and Russell Greiner and Abram Hindle},
booktitle = {ICSE},
year = {2018},
doi = {10.1145/3180155.3182535}
}