一句话：通过对22名工业界DL开发者的半结构化访谈，揭示演化DL系统测试的挑战、实践与需求。
深度学习系统已广泛部署于自动驾驶、智能医疗等领域，与传统的软件一样，它们也需要持续演化以应对不断变化的用户需求，但如何保障这些持续演化系统的质量，尤其在测试环节，仍面临巨大挑战。为理解工业界开发者如何应对这些挑战以及他们遭遇的额外障碍，本文对来自不同领域和背景的22名深度学习开发者进行了半结构化访谈，重点关注三方面：测试演化DL系统时遇到的困难、所采用的实践解决方案，以及对额外支持工具的期望。研究结果揭示了回归故障、线上线下差异、测试数据收集等测试难点，总结了开发者应对这些挑战的最佳实践，并指出了提升演化深度学习系统测试有效性的潜在未来研究方向。
Deep Learning (DL) systems have been widely adopted across various industrial domains such as autonomous driving and intelligent healthcare. As with traditional software, DL systems also need to constantly evolve to meet ever-changing user requirements. However, ensuring the quality of these continuously evolving systems presents significant challenges, especially in the context of testing. Understanding how industry developers address these challenges and what extra obstacles they are facing could provide valuable insights for further safeguarding the quality of DL systems. To reach this goal, we conducted semi-structured interviews with 22 DL developers from diverse domains and backgrounds. More specifically, our study focuses on exploring the challenges developers encounter in testing evolving DL systems, the practical solutions they employ, and their expectations for extra support. Our results highlight the difficulties in testing evolving DL systems (e.g., regression faults, online-offline differences, and test data collection) and identify the best practices for D L developers to address these challenges. Additionally, we pinpoint potential future research directions to enhance testing effectiveness in evolving DL systems.
DOI 原文 ·
@inproceedings{paperbot8749,
title = {Navigating the Testing of Evolving Deep Learning Systems: An Exploratory Interview Study},
author = {Hanmo You and Zan Wang and Bin Lin and Junjie Chen},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00106}
}