一句话：通过大规模实证研究揭示实用日志异常检测的挑战，并提出集成学习系统 LogAD，F1 达 0.83。
在线服务系统的日志数据蕴含丰富运行信息，日志异常检测对服务可靠性工程至关重要。然而现有方法在实际部署时存在局限：难以处理多样日志与复杂异常模式、可解释性差、缺乏领域知识。为量化这些挑战，本文基于大规模真实数据开展首个实证与实验研究，发现富信息日志呈现关键词、模板计数、模板序列、变量取值与变量分布等多种异常模式，而现有方法应对复杂模式效果不佳。受此启发，作者提出通用系统 LogAD，基于集成学习融合多种异常检测方法与领域知识。在真实数据上平均 F1 达 0.83，优于所有基线，并总结了成功案例与经验教训。
Log data is an essential and valuable resource of online service systems, which records detailed information of system running status and user behavior. Log anomaly detection is vital for service reliability engineering, which has been extensively studied. However, we find that existing approaches suffer from several limitations when deploying them into practice, including 1) inability to deal with various logs and complex log abnormal patterns; 2) poor interpretability; 3) lack of domain knowledge. To help understand these practical challenges and investigate the practical performance of existing work quantitatively, we conduct the first empirical study and an experimental study based on large-scale real-world data. We find that logs with rich information indeed exhibit diverse abnormal patterns (e.g., keywords, template count, template sequence, variable value, and variable distribution). However, existing approaches fail to tackle such complex abnormal patterns, producing unsatisfactory performance. Motivated by obtained findings, we propose a generic log anomaly detection system named LogAD based on ensemble learning, which integrates multiple anomaly detection approaches and domain knowledge, so as to handle complex situations in practice. About the effectiveness of LogAD, the average F1-score achieves 0.83, outperforming all baselines. Besides, we also share some success cases and lessons learned during our study. To our best knowledge, we are the first to investigate practical log anomaly detection in the real world deeply. Our work is helpful for practitioners and researchers to apply log anomaly detection to practice to enhance service reliability.
DOI 原文 ·
@inproceedings{paperbot6388,
title = {An empirical investigation of practical log anomaly detection for online service systems},
author = {Nengwen Zhao and Honglin Wang and Zeyan Li and Xiao Peng and Gang Wang and Zhu Pan and Yong Wu and Zhen Feng and Xidao Wen and Wenchi Zhang and Kaixin Sui and Dan Pei},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3473933}
}