一句话：提出 Onion，通过日志团簇与对比分析自动精准定位云系统中指示故障的日志，兼顾效果与效率。
云系统中故障会影响服务可用性，需要快速缓解；运维和开发人员通常检查日志进行故障诊断，但日志规模大、类型杂、细节多，人工定位指示故障的日志耗时且易错。本文提出 Onion 自动方案，首先给出定位 incident-indicating logs 的三条准则：一致性、影响与双边差异；进而提出 log clique 作为满足这些准则的日志聚集体，并通过 incident-aware 日志表示与渐进式日志聚类获得 clique，再对 clique 做对比分析以识别指示日志。在带标注日志数据集上，Onion 平均 F1 达 0.95，可在几分钟内处理数百万条日志；并已成功应用于 Microsoft 云系统，真实故障案例的定量与定性分析验证了实用性。
In cloud systems, incidents affect the availability of services and require quick mitigation actions. Once an incident occurs, operators and developers often examine logs to perform fault diagnosis. However, the large volume of diverse logs and the overwhelming details in log data make the manual diagnosis process time-consuming and error-prone. In this paper, we propose Onion, an automatic solution for precisely and efficiently locating incident-indicating logs, which can provide useful clues for diagnosing the incidents. We first point out three criteria for localizing incident-indicating logs, i.e., Consistency, Impact, and Bilateral-Difference. Then we propose a novel agglomeration of logs, called log clique, based on which these criteria are satisfied. To obtain log cliques, we develop an incident-aware log representation and a progressive log clustering technique. Contrast analysis is then performed on the cliques to identify the incident-indicating logs. We have evaluated Onion using well-labeled log datasets. Onion achieves an average F1-score of 0.95 and can process millions of logs in only a few minutes, demonstrating its effectiveness and efficiency. Onion has also been successfully applied to the cloud system of Microsoft. Its practicability has been confirmed through the quantitative and qualitative analysis of the real incident cases.
DOI 原文 ·
@inproceedings{paperbot6357,
title = {Onion: identifying incident-indicating logs for cloud systems},
author = {Xu Zhang and Yong Xu and Si Qin and Shilin He and Bo Qiao and Ze Li and Hongyu Zhang and Xukun Li and Yingnong Dang and Qingwei Lin and Murali Chintalapati and Saravanakumar Rajmohan and Dongmei Zhang},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3473919}
}