一句话：提出 InferLog，通过前缀感知的 ICL 精炼与元学习配置调优，在不损失准确率的前提下加速在线日志解析的 LLM 推理。
现代软件系统产生海量运行时日志，需要高效准确的在线日志解析支撑异常检测与根因分析。LLM 在日志解析上精度领先，但生产部署面临隐私风险（需本地部署）以及在线解析对延迟与吞吐的严苛要求；已有方法虽能减少 LLM 查询次数，却忽视了 LLM 推理本身的固有开销——并发解析请求会造成推理系统性能退化。本文提出 InferLog，首个面向在线日志解析的 LLM 推理优化方法。其核心洞察是：在基于 LLM 的在线日志解析中，推理效率而非解析精度才是关键瓶颈。InferLog 通过两项设计加速推理：(i) 前缀感知的 ICL 精炼策略，重写 in-context learning 示例内容与排列以提升前缀缓存效率；(ii) 基于元学习的快速、任务特定配置调优流水线，搜索最优的 LLM 推理系统配置。基于 Loghub-2k 数据集与 vLLM 的实验表明，InferLog 显著优于现有推理优化方法，并在不牺牲解析精度的前提下大幅加速当前最优的 LLM 日志解析器。
Modern software systems generate massive volumes of runtime logs, necessitating efficient and accurate log parsing to enable critical downstream tasks such as anomaly detection and root cause analysis. Recently, large language models (LLMs) have achieved advanced accuracy on log parsing, but their deployment in production environments faces two major limitations: First, the privacy risks associated with commercial LLMs, driving the adoption of local deployment. Second, online log parsing poses stringent challenges for latency and throughput. While recent methods reduce the number of LLM queries, they overlook the inherent overhead of LLM inference where concurrent log parsing requests can lead to performance degradation in the LLM inference system. In this study, we present InferLog, the first LLM inference optimization method for online log parsing. Our key insight is that the inference efficiency emerges as the vital bottleneck in LLM-based online log parsing, rather than parsing accuracy. InferLog accelerates inference by designing (i) A prefix-aware ICL refinement strategy to refine the examples and permutation of in-context learning to improve the prefix caching efficiency. (ii) A rapid and task-specific configuration tuning pipeline based on meta-learning to find the optimal LLM inference system configuration. The experimental results based on Loghub-2k dataset and vLLM demonstrate that InferLog significantly outperforms existing inference optimization methods and markedly accelerates the state-of-the-art LLM-based log parsers without compromising parsing accuracy.
@inproceedings{abs-2507-08523,
title = {InferLog: Accelerating LLM Inference for Online Log Parsing via ICL-oriented Prefix Caching},
author = {Yilun Wang and Pengfei Chen and Haiyu Huang and Zilong He and Gou Tan and Chuanfu Zhang and Jingkai He and Zibin Zheng},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3764523}
}