一句话：通过对微软23个开发者机器人、1.3万仓库及访谈调研的实证分析，揭示其动机、效用感知与挑战。
论文关注 CI/CD 中开发者机器人被用于“左移”和提升开发效率，但缺乏大规模实证理解。作者在微软开展案例研究，识别并分析部署于 13,000 个仓库、每日服务约 6,000 名开发者的 23 个机器人，将其分为配置违规、安全、数据隐私、开发者生产力和代码质量五类；结合对机器人开发者和用户的访谈与问卷，以及约 50 万条、跨度一年半的历史机器人动作，归纳引入机器人的软件工作流、影响用户感知有用性的因素和采用中的挑战。结果复现噪声等既有问题，并揭示跨团队沟通等新收益和机器人过多等新挑战。
Continuous integration and deployment (CI/CD) is now a widely adopted development model in practice as it reduces the time from ideas to customers. This adoption has also revived the idea of "shifting left" during software development -- a practice intended to find and prevent defects early in the software delivery process. To assist with that, engineering systems integrate developer bots in the development workflow to improve developer productivity and help them identify issues early in the software delivery process. In this paper, we present a case study of developer bots in Microsoft. We identify and analyze 23 developer bots that are deployed across 13,000 repositories and assist about 6,000 developers daily in their CI/CD software development workflows. We classify these bots across five major categories: Config Violation, Security, Data-privacy, Developer Productivity, and Code Quality. By conducting interviews and surveys with bot developers and bot users and by analyzing about half a million historical bot actions spanning over one and a half years, we present software workflows that motivate bot instrumentation, factors impacting their usefulness as perceived by bot users, and challenges associated with their use. Our findings echo existing issues with bots, such as noise, and illustrate new benefits (e.g., cross-team communication) and challenges (e.g., too many bots) for large software teams.
@inproceedings{AsthanaSVAH23,
title = {A Case Study of Developer Bots: Motivations, Perceptions, and Challenges},
author = {Sumit Asthana and Hitesh Sajnani and Elena Voyloshnikova and Birendra Acharya and Kim Herzig},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3616248}
}