一句话：构建262个开源ML产品数据集，并分析30个仓库揭示从模型到产品的开发实践与架构。
该研究关注机器学习组件进入面向用户软件产品时从原型到产品的转化挑战，以及商业机器学习产品源码难获取导致研究受限的问题。作者从GitHub超过50万个机器学习相关项目中识别出262个面向最终用户的开源机器学习产品，构建数据集，并对其中30个进行定性与定量分析，回答六个开发实践与系统架构问题。结果发现多数产品更接近创业式开发；共21项发现，包括数据科学家参与有限、机器学习与非机器学习代码模块化异常低、模型集成架构多样，以及模型测试、流水线自动化和监控等最佳实践普及有限。最后讨论对研究、开发与教育的七点启示，如支持无数据科学家团队的工具、教育机会与隐私保护遥测。
Machine learning (ML) components are increasingly incorporated into software products for end-users, but developers face challenges in transitioning from ML prototypes to products. Academics have limited access to the source of commercial ML products, hindering research progress to address these challenges. In this study, first and foremost, we contribute a dataset of 262 open-source ML products for end users (not just models), identified among more than half a million ML-related projects on GitHub. Then, we qualitatively and quantitatively analyze 30 open-source ML products to answer six broad research questions about development practices and system architecture. We find that the majority of the ML products in our sample represent more startup-style development than reported in past interview studies. We report 21 findings, including limited involvement of data scientists in many open-source ML products, unusually low modularity between ML and non-ML code, diverse architectural choices on incorporating models into products, and limited prevalence of industry best practices such as model testing, pipeline automation, and monitoring. Additionally, we discuss seven implications of this study on research, development, and education, including the need for tools to assist teams without data scientists, education opportunities, and open-source-specific research for privacy-preserving telemetry.
DOI 原文 ·
@inproceedings{paperbot8709,
title = {The Product Beyond the Model - An Empirical Study of Repositories of Open-Source ML Products},
author = {Nadia Nahar and Haoran Zhang and Grace Lewis and Shurui Zhou and Christian Kästner},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00006}
}