一句话：通过分析26个开源ML项目与327个补丁，揭示ML系统重构动因、技术债类别及14/7个新类别。
机器学习系统由模型与众多学习支撑子系统组成，结构复杂，长期演进中容易积累经典及ML特有的技术债，但关于其实际演化与维护的知识仍不足。本文对26个开源项目、约420万行代码及327个人工检查的代码补丁开展实证研究，分析开发者执行的重构及其缓解的技术债问题。结果表明：重构动因既有ML特定原因也有一般原因；部分重构对应已有技术债类别，部分则不能；代码重复是贯穿性问题，尤其涉及ML配置与模型代码，而后者也是重构最频繁的部分。作者进一步提出14种新的ML特定重构和7类新的ML特定技术债，并给出建议、最佳实践与反模式。
Machine Learning (ML), including Deep Learning (DL), systems, i.e., those with ML capabilities, are pervasive in today's data-driven society. Such systems are complex; they are comprised of ML models and many subsystems that support learning processes. As with other complex systems, ML systems are prone to classic technical debt issues, especially when such systems are long-lived, but they also exhibit debt specific to these systems. Unfortunately, there is a gap of knowledge in how ML systems actually evolve and are maintained. In this paper, we fill this gap by studying refactorings, i.e., source-to-source semantics-preserving program transformations, performed in real-world, open-source software, and the technical debt issues they alleviate. We analyzed 26 projects, consisting of 4.2 MLOC, along with 327 manually examined code patches. The results indicate that developers refactor these systems for a variety of reasons, both specific and tangential to ML, some refactorings correspond to established technical debt categories, while others do not, and code duplication is a major cross-cutting theme that particularly involved ML configuration and model code, which was also the most refactored. We also introduce 14 and 7 new ML-specific refactorings and technical debt categories, respectively, and put forth several recommendations, best practices, and anti-patterns. The results can potentially assist practitioners, tool developers, and educators in facilitating long-term ML system usefulness.
@inproceedings{TangK0SSR21,
title = {An Empirical Study of Refactorings and Technical Debt in Machine Learning Systems},
author = {Yiming Tang and Raffi Khatchadourian and Mehdi Bagherzadeh and Rhia Singh and Ajani Stewart and Anita Raja},
booktitle = {ICSE},
year = {2021},
doi = {10.1109/icse43902.2021.00033}
}