一句话：提出无需日志解析的 NeuralLog，直接语义向量化原始日志并用 Transformer 检测异常，四个数据集 F1 超 0.95。
现有基于日志的异常检测方法通常依赖日志解析，但解析错误（如 OOV 词和语义误解）会丢失关键信息，显著影响检测效果。本文提出 NeuralLog，一种无需日志解析的异常检测方法。它直接提取原始日志消息的语义含义，将其表示为语义向量，并通过基于 Transformer 的分类模型利用日志序列的上下文信息检测异常。作者在四个公开数据集上评估，NeuralLog 的 F1 分数均超过 0.95，优于现有方法，表明其能有效理解日志语义并实现准确异常检测。
Software systems often record important runtime information in system logs for troubleshooting purposes. There have been many studies that use log data to construct machine learning models for detecting system anomalies. Through our empirical study, we find that existing log-based anomaly detection approaches are significantly affected by log parsing errors that are introduced by 1) OOV (out-of-vocabulary) words, and 2) semantic misunderstandings. The log parsing errors could cause the loss of important information for anomaly detection. To address the limitations of existing methods, we propose NeuralLog, a novel log-based anomaly detection approach that does not require log parsing. NeuralLog extracts the semantic meaning of raw log messages and represents them as semantic vectors. These representation vectors are then used to detect anomalies through a Transformer-based classification model, which can capture the contextual information from log sequences. Our experimental results show that the proposed approach can effectively understand the semantic meaning of log messages and achieve accurate anomaly detection results. Overall, NeuralLog achieves F1-scores greater than 0.95 on four public datasets, outperforming the existing approaches.
DOI 原文 ·
@inproceedings{paperbot6577,
title = {Log-based Anomaly Detection Without Log Parsing},
author = {Van-Hoang Le and Hongyu Zhang},
booktitle = {ASE},
year = {2021},
doi = {10.1109/ase51524.2021.9678773}
}