一句话：提出 MetaLog，用全局语义嵌入与元学习实现跨系统日志异常检测，仅需目标系统 1% 标签即可达到 80%+ F1。
日志异常检测对保障软件稳定性至关重要，但现有方法通常依赖大量目标系统的标注历史数据，而许多真实系统缺乏此类数据。本文提出 MetaLog，一种可泛化的跨系统日志异常检测方法。它首先通过全局一致的语义嵌入模块，将日志事件映射到共享全局空间的语义向量；随后利用元学习范式提升模型在标签稀缺目标系统上的泛化能力。作者在 HDFS、BGL、OpenStack 和 Thunderbird 四个公开日志数据集上评估。结果表明，仅使用目标系统 1% 的标注日志，MetaLog 的 F1 分数超过 80%，与使用 100% 标注数据训练的最先进监督模型性能相近；在同样 1% 标注设置下，比最先进的基于迁移学习的跨系统异常检测模型高出 20%。
Log-based anomaly detection plays a crucial role in ensuring the stability of software. However, current approaches for log-based anomaly detection heavily depend on a vast amount of labeled historical data, which is often unavailable in many real-world systems. To mitigate this problem, we leverage the features of the abundant historical labeled logs of mature systems to help construct anomaly detection models of new systems with very few labels, that is, to generalize the model ability trained from labeled logs of mature systems to achieve anomaly detection on new systems with insufficient data labels. Specifically, we propose MetaLog, a generalizable cross-system anomaly detection approach. MetaLog first incorporates a globally consistent semantic embedding module to obtain log event semantic embedding vectors in a shared global space. Then it leverages the meta-learning paradigm to improve the model's generalization ability. We evaluate MetaLog's performance on four public log datasets (HDFS, BGL, OpenStack, and Thunderbird) from four different systems. Results show that MetaLog reaches over 80% F1-score when using only 1% labeled logs of the target system, showing similar performance with state-of-the-art supervised anomaly detection models trained with 100% labeled data. Besides, it outperforms state-of-art transfer-learning-based cross-system anomaly detection models by 20% in the same settings of 1% labeled training logs of the target system.
@inproceedings{paperbot11358,
title = {MetaLog: Generalizable Cross-System Anomaly Detection from Logs with Meta-Learning},
author = {Chenyangguang Zhang and Tong Jia and Guopeng Shen and Pinyan Zhu and Ying Li},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639205}
}