一句话：提出 SCWarn，用多模态学习从异构多源数据中识别在线服务系统的坏变更并生成可解释告警。
大规模在线服务系统中软件变更频繁，可能引入代码或配置问题并导致事故。为理解坏变更，作者基于某大型商业银行真实数据开展首个实证研究，发现约50.4%的事故由坏变更引起，主因包括代码缺陷、配置错误、资源竞争和软件版本，且现有检测难以处理异构多源数据。受此启发，论文提出 SCWarn，利用多模态学习从异构多源数据中识别异常，准确及时地产生可解释告警。在含多种坏变更的两个数据集上，SCWarn 平均 F1 达 0.95，MTTD 降低 20.4%～60.7%，并分享了实际应用中的成功案例与经验。
In large-scale online service systems, software changes are inevitable and frequent. Due to importing new code or configurations, changes are likely to incur incidents and destroy user experience. Thus it is essential for engineers to identify bad software changes, so as to reduce the influence of incidents and improve system re- liability. To better understand bad software changes, we perform the first empirical study based on large-scale real-world data from a large commercial bank. Our quantitative analyses indicate that about 50.4% of incidents are caused by bad changes, mainly be- cause of code defect, configuration error, resource contention, and software version. Besides, our qualitative analyses show that the current practice of detecting bad software changes performs not well to handle heterogeneous multi-source data involved in soft- ware changes. Based on the findings and motivation obtained from the empirical study, we propose a novel approach named SCWarn aiming to identify bad changes and produce interpretable alerts accurately and timely. The key idea of SCWarn is drawing support from multimodal learning to identify anomalies from heterogeneous multi-source data. An extensive study on two datasets with various bad software changes demonstrates our approach significantly outperforms all the compared approaches, achieving 0.95 F1-score on average and reducing MTTD (mean time to detect) by 20.4%∼60.7%. In particular, we shared some success stories and lessons learned from the practical usage.
DOI 原文 ·
@inproceedings{paperbot6442,
title = {Identifying bad software changes via multimodal anomaly detection for online service systems},
author = {Nengwen Zhao and Junjie Chen and Zhaoyang Yu and Honglin Wang and Jiesong Li and Bin Qiu and Hongyu Xu and Wenchi Zhang and Kaixin Sui and Dan Pei},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3468543}
}