一句话:提出 LogPipe,通过动态知识库增强 LLM 日志异常检测,提升准确率并降低成本与增强可解释性。
现有基于大语言模型(LLM)的日志异常检测方法存在准确率有限、计算成本高、可解释性差的问题。论文提出 LogPipe 框架,利用离散与语义日志模式构建动态知识库,并通过情感词典和频繁模式挖掘生成动态模式;推理时将日志序列与知识库匹配,为 LLM 提供具体指导,从而提升检测准确率并生成详细解释。持续更新机制保持知识库时效性,同时减少冗余 LLM 查询,显著降低推理成本。在八个公开数据集上,LogPipe 平均 F1 达到 0.975,优于当前最优模型,并降低 token 消耗;在故障定位方面也表现优异,增强了异常检测的可解释性。
Log anomaly detection is critical for maintaining system reliability, yet existing large language model (LLM)-based methods suffer from limited accuracy, high computational costs, and poor explainability. In this paper, we introduce LogPipe, a novel framework that enhances LLM-based log anomaly detection by integrating a dynamic knowledge base. LogPipe constructs a knowledge base using discrete and semantic log patterns, augmented by dynamic patterns that are generated by a sentiment dictionary and frequent pattern mining. During inference, log sequences are matched against the knowledge base to provide specific guidance to the LLM, improving detection accuracy and generating detailed explanations. A continuous update mechanism ensures that the knowledge base remains relevant while minimizing redundant LLM queries, significantly reducing inference costs. Evaluated on eight public datasets, LogPipe achieves an average F1 score of 0.975, outperforming state-of-the-art models, with reduced token consumption. Additionally, LogPipe excels in fault localization, which enhances the explainability of detected anomalies.