一句话：提出知识增强预训练语言模型 KnowLog，利用缩略语、文档描述与对比学习提升日志理解。
现有预训练语言模型在日志理解上存在三个不足：难以理解领域术语尤其缩写、难以充分捕获完整日志上下文、难以获得同一日志不同风格的通用表示。为此，本文提出 KnowLog，从公开文档中提取日志缩略语与自然语言描述，分别作为局部知识和全局知识，并设计新的预训练任务将知识注入模型；同时设计基于对比学习的预训练任务以获得通用表示。作者在六个日志理解任务上微调评估，实验表明 KnowLog 显著提升日志理解效果，优于无知识增强的预训练语言模型并取得最优结果；迁移学习和低资源场景下的额外实验也展示了其明显优势。
Logs as semi-structured text are rich in semantic information, making their comprehensive understanding crucial for automated log analysis. With the recent success of pre-trained language models in natural language processing, many studies have leveraged these models to understand logs. Despite their successes, existing pre-trained language models still suffer from three weaknesses. Firstly, these models fail to understand domain-specific terminology, especially abbreviations. Secondly, these models struggle to adequately capture the complete log context information. Thirdly, these models have difficulty in obtaining universal representations of different styles of the same logs. To address these challenges, we introduce KnowLog, a knowledge-enhanced pre-trained language model for log understanding. Specifically, to solve the previous two challenges, we exploit abbreviations and natural language descriptions of logs from public documentation as local and global knowledge, respectively, and leverage this knowledge by designing novel pre-training tasks for enhancing the model. To solve the last challenge, we design a contrastive learning-based pre-training task to obtain universal representations. We evaluate KnowLog by fine-tuning it on six different log understanding tasks. Extensive experiments demonstrate that KnowLog significantly enhances log understanding and achieves state-of-the-art results compared to existing pre-trained language models without knowledge enhancement. Moreover, we conduct additional experiments in transfer learning and low-resource scenarios, showcasing the substantial advantages of KnowLog. Our source code and detailed experimental data are available at https://github.com/LeaperOvO/KnowLog.
DOI 原文 ·
@inproceedings{paperbot11474,
title = {KnowLog: Knowledge Enhanced Pre-trained Language Model for Log Understanding},
author = {Lipeng Ma and Weidong Yang and Bo Xu and Sihang Jiang and Ben Fei and Jiaqing Liang and Mingjie Zhou and Yanghua Xiao},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3623304}
}