一句话：首次对 Apollo 自动驾驶系统进行案例研究，揭示 ML 模型与代码逻辑的集成方式及测试不足。
自动驾驶系统（ADS）是最大规模、最有价值的机器学习（ML）赋能系统之一，但学界对其内部 ML 模型的角色、特殊架构及与非平凡代码逻辑的集成关系仍缺乏全面的实证理解。本文对当前最先进的 ADS 之一 Apollo 开展深入案例研究。作者分析 Apollo 源代码，呈现其底层 ML 模型系统架构，揭示多个 ML 模型之间如何交互、模型如何与代码逻辑集成形成复杂系统。最后从动态视角检查 Apollo，发现模型相关组件被大量使用，但总体缺乏充分测试。研究揭示了复杂 ML 赋能系统的潜在维护挑战，并指出了改进 ADS 及通用 ML 系统质量保障的未来方向。
Autonomous Driving System (ADS) is one of the most promising and valuable large-scale machine learning (ML) powered systems. Hence, ADS has attracted much attention from academia and practitioners in recent years. Despite extensive study on ML models, it still lacks a comprehensive empirical study towards understanding the ML model roles, peculiar architecture, and complexity of ADS (i.e., various ML models and their relationship with non-trivial code logic). In this paper, we conduct an in-depth case study on Apollo, which is one of the state-of-the-art ADS, widely adopted by major automakers worldwide. We took the first step to reveal the integration of the underlying ML models and code logic in Apollo. In particular, we study the Apollo source code and present the underlying ML model system architecture. We present our findings on how the ML models interact with each other, and how the ML models are integrated with code logic to form a complex system. Finally, we inspect Apollo in a dynamic view and notice the heavy use of model-relevant components and the lack of adequate tests in general. Our study reveals potential maintenance challenges of complex ML-powered systems and identifies future directions to improve the quality assurance of ADS and general ML systems.
DOI 原文 ·
@inproceedings{paperbot5785,
title = {A first look at the integration of machine learning models in complex autonomous driving systems: a case study on Apollo},
author = {Zi Peng and Jinqiu Yang and Tse-Hsun (Peter) Chen and Lei Ma},
booktitle = {FSE},
year = {2020},
doi = {10.1145/3368089.3417063}
}