paperbot · PL 论文追踪

全部论文 待读 收藏

P2 LogSage: An LLM-Based Framework for CI/CD Failure Detection and Remediation with Industrial Validation

LogSage：一个基于 LLM 的 CI/CD 故障检测与修复框架及工业验证
ASE 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE)2025匹配度 3/10 · 该论文属于 LLM 驱动的 DevOps/CI-CD 日志分析与故障修复，是 AI for SE 的工程应用，与用户的核心方向（静态分析应用于 Agentic 程序、LLM 辅助静态分析/形式化验证、浮点程序分析、贝叶斯程序分析）均无实质交集，且研究目标偏向 LLM 框架自身的提示与 RAG 能力。仅在'工业级部署验证'和'在缺乏 benchmark 的新场景构建 benchmark'两点上契合用户偏好。引用 5 inbox
Weiyuan Xu, Juntao Luo, Tao Huang, Kaixin Sui, Jie Geng, Qijun Ma, Isami Akasaka, Xiaoxue Shi, Jing Tang, Peng Cai
机器学习与程序语言测试、调试与修复
crossref: 2026-09-12

AI 速览

一句话：提出首个端到端 LLM 驱动的 CI/CD 故障根因分析与自动修复框架，并完成一年工业级部署验证。

CI/CD 流水线的失败诊断与修复复杂且耗费人力。LogSage 是首个端到端的 LLM 驱动框架，用于 CI/CD 失败的根因分析（RCA）与自动修复：它先用 token 高效的日志预处理管线过滤噪声、抽取关键错误，再以结构化诊断提示完成准确 RCA；在解决方案生成阶段，借助检索增强生成（RAG）复用历史修复经验，并通过 LLM 工具调用执行自动化修复。作者构建了包含 367 个 GitHub CI/CD 失败的新基准，RCA 阶段精度超过 98%、召回近乎完美，F1 较近期 LLM 基线提升逾 38 个百分点；在字节跳动为期一年的工业部署中处理超过 107 万次执行，端到端精度超过 80%，表明该方案可规模化落地于真实 DevOps 流程。

问题
CI/CD 流水线失败频繁且诊断与修复高度依赖人工，现有 LLM 方法在日志噪声、根因定位与修复执行上难以端到端落地。
方法
采用 token 高效的日志预处理抽取关键错误，用结构化诊断提示做根因分析，并以 RAG 复用历史修复、通过 LLM 工具调用执行自动化修复。
评估
在新建的 367 个 GitHub CI/CD 失败基准上与近期 LLM 基线对比 RCA 精度、召回与 F1，并报告在字节跳动一年、超 107 万次执行的工业部署端到端精度。
原文摘要（Abstract）

Continuous Integration and Deployment (CI/CD) pipelines are critical to modern software engineering, yet diagnosing and resolving their failures remains complex and labor-intensive. We present LogSage, the first end-to-end LLM-powered framework for root cause analysis (RCA) and automated remediation of CI/CD failures. LogSage employs a token-efficient log preprocessing pipeline to filter noise and extract critical errors, then performs structured diagnostic prompting for accurate RCA. For solution generation, it leverages retrieval-augmented generation (RAG) to reuse historical fixes and invokes automation fixes via LLM tool-calling.On a newly curated benchmark of 367 GitHub CI/CD failures, LogSage achieves over 98% precision, near-perfect recall, and an F1 improvement of more than 38% points in the RCA stage, compared with recent LLM-based baselines. In a yearlong industrial deployment at ByteDance, it processed over 1.07M executions, with end-to-end precision exceeding 80%. These results demonstrate that LogSage provides a scalable and practical solution for automating CI/CD failure management in real-world DevOps workflows.

链接与引用

DOI 原文 ·

BibTeX 
@inproceedings{paperbot9491,
  title = {LogSage: An LLM-Based Framework for CI/CD Failure Detection and Remediation with Industrial Validation},
  author = {Weiyuan Xu and Juntao Luo and Tao Huang and Kaixin Sui and Jie Geng and Qijun Ma and Isami Akasaka and Xiaoxue Shi and Jing Tang and Peng Cai},
  booktitle = {ASE},
  year = {2025},
  doi = {10.1109/ase63991.2025.00310}
}