paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Log Parsing: How Far Can ChatGPT Go?

日志解析:ChatGPT 能走多远?
ASE 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)2023匹配度 1/10 · 该论文研究 ChatGPT 在日志解析任务中的表现与提示方法,不属于静态分析、形式化验证、浮点程序分析或贝叶斯程序分析,且更偏向评估 LLM 本身能力,与用户画像核心方向不匹配。引用 64 inbox
Van-Hoang Le, Hongyu Zhang
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:系统评估 ChatGPT 在日志解析任务中的表现,并分析不同提示方法的影响。

软件日志是大规模系统可靠性与可维护性的关键运行信息,日志解析则是将原始日志消息转换为结构化数据、支撑后续日志分析的重要步骤。然而,当前先进大语言模型 ChatGPT 在自动日志解析中的表现尚不明确。本文围绕两个研究问题展开评估:(1)ChatGPT 能否有效解析日志;(2)不同提示方法下 ChatGPT 表现如何。结果表明,在合适提示尤其是 few-shot prompting 下,ChatGPT 能取得有前景的日志解析效果;作者据此总结了基于 ChatGPT 的日志解析所面临的挑战与机会。

问题
评估 ChatGPT 在自动日志解析中的能力,以及不同提示方法对其性能的影响。
方法
通过设计并比较多种提示方法(尤其是 few-shot prompting)对 ChatGPT 进行日志解析实验评估。
评估
以日志解析效果为核心指标,回答 ChatGPT 能否有效解析日志及不同提示方法表现如何两个研究问题。
原文摘要(Abstract)

Software logs play an essential role in ensuring the reliability and maintainability of large-scale software systems, as they are often the sole source of runtime information. Log parsing, which converts raw log messages into structured data, is an important initial step towards downstream log analytics. In recent studies, ChatGPT, the current cutting-edge large language model (LLM), has been widely applied to a wide range of software engineering tasks. However, its performance in automated log parsing remains unclear. In this paper, we evaluate ChatGPT's ability to undertake log parsing by addressing two research questions. (1) Can ChatGPT effectively parse logs? (2) How does ChatGPT perform with different prompting methods? Our results show that ChatGPT can achieve promising results for log parsing with appropriate prompts, especially with few-shot prompting. Based on our findings, we outline several challenges and opportunities for ChatGPT-based log parsing.

链接与引用

DOI 原文 · arXiv · PDF(开放获取) · DBLP

BibTeX
@inproceedings{LeZ23,
  title = {Log Parsing: How Far Can ChatGPT Go?},
  author = {Van-Hoang Le and Hongyu Zhang},
  booktitle = {ASE},
  year = {2023},
  doi = {10.1109/ase56229.2023.00206}
}