一句话:提出无需演示示例的 LLM 日志解析器 LogBatcher,借聚类、缓存匹配与批处理提示降低调用开销。
日志解析将原始日志消息转为结构化格式,是大规模软件系统日志自动化分析的重要初始步骤。传统方法依赖启发式或手工特征,跨日志源泛化差且常需大量调参;近期基于大语言模型的解析器虽具生成能力,却依赖演示示例,导致 LLM 调用开销很高。本文提出 LogBatcher,一种无需训练、无需标注数据且免除演示示例的高性价比 LLM 日志解析器。它先通过聚类将日志划分为若干分区,再利用缓存匹配将日志与已解析的日志模板对齐,最后对每个分区中的一组日志进行批处理,为 LLM 提供更相关的提示上下文。作者在 16 个公开日志数据集上评估,结果表明 LogBatcher 在有效性和效率上均表现良好。
Log parsing, the process of converting raw log messages into structured formats, is an important initial step for automated analysis of logs of large-scale software systems. Traditional log parsers often rely on heuristics or handcrafted features, which may not generalize well across diverse log sources or require extensive model tuning. Recently, some log parsers have utilized powerful generative capabilities of large language models (LLMs). However, they heavily rely on demonstration examples, resulting in substantial overhead in LLM invocations. To address these issues, we propose LogBatcher, a cost-effective LLM-based log parser that requires no training process or labeled data. To leverage latent characteristics of log data and reduce the overhead, we divide logs into several partitions through clustering. Then we perform a cache matching process to match logs with previously parsed log templates. Finally, we provide LLMs with better prompt context specialized for log parsing by batching a group of logs from each partition. We have conducted experiments on 16 public log datasets and the results show that LogBatcher is effective and efficient for log parsing.