一句话：提出 Log3T，通过变量生成模仿与测试时训练更新，使日志解析器泛化到新日志类型。
日志解析将半结构化日志转为结构化日志，是自动化日志分析的第一步。实际系统中新来的日志常含新日志类型：低频系统状态对应的类型难以被收集，系统更新也会引入新类型；现有解析器还依赖基于先验知识的预处理，且参数难以泛化到新类型。本文提出变量生成模仿策略，并构建 Log3T：用预训练 Transformer 编码器提取日志模板，并通过改进的测试时训练在解析时更新参数，以适应新日志类型。在 16 个基准数据集上，Log3T 的解析准确率优于当前最优解析器，并能自动适应新来日志中的新日志类型。
Log parsing, which converts semi-structured logs into structured logs, is the first step for automated log analysis. Existing parsers are still unsatisfactory in real-world systems due to new log types in new-coming logs. In practice, available logs collected during system runtime often do not contain all the possible log types of a system because log types related to infrequently activated system states are unlikely to be recorded and new log types are frequently introduced with system updates. Meanwhile, most existing parsers require preprocessing to extract variables in advance, but preprocessing is based on the operator’s prior knowledge of available logs and therefore may not work well on new log types. In addition, parser parameters set based on available logs are difficult to generalize to new log types. To support new log types, we propose a variable generation imitation strategy to craft a novel log parsing approach with generalization ability, called Log3T. Log3T employs a pre-trained transformer encoder-based model to extract log templates and can update parameters at parsing time to adapt to new log types by a modified test-time training. Experimental results on 16 benchmark datasets show that Log3T outperforms the state-of-the-art parsers in terms of parsing accuracy. In addition, Log3T can automatically adapt to new log types in new-coming logs.
@inproceedings{YuW0HCL23,
title = {Log Parsing with Generalization Ability under New Log Types},
author = {Siyu Yu and Yifan Wu and Zhijing Li and Pinjia He and Ningjiang Chen and Changjian Liu},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3616355}
}