一句话:提出 AlertGuardian,融合 LLM 与图模型,通过去噪、摘要与规则精化三阶段优化云系统告警生命周期。
大规模云系统告警数量爆炸,导致告警疲劳并降低运维效率。本文提出 AlertGuardian,协同大语言模型与轻量图模型优化告警生命周期,含三阶段:告警去噪利用图学习模型与虚拟噪声过滤噪声;告警摘要采用检索增强生成(RAG)与 LLM 生成可操作摘要;告警规则精化借助多智能体迭代反馈提升规则质量。在 Company-X 四个真实服务数据集上评估,告警减少 94.8%,诊断准确率达 90.5%,并改进 1,174 条告警规则,其中 375 条被 SRE 接受(32%),最后总结部署经验与教训。
Alerts are critical for detecting anomalies in large-scale cloud systems, ensuring reliability and user experience. However, current systems generate overwhelming volumes of alerts, degrading operational efficiency due to ineffective alert life-cycle management. This paper details the efforts of Company-X to optimize alert life-cycle management, addressing alert fatigue in cloud systems. We propose AlertGuardian, a framework collaborating large language models (LLMs) and lightweight graph models to optimize the alert life-cycle through three phases: Alert Denoise uses graph learning model with virtual noise to filter noise, Alert Summary employs Retrieval Augmented Generation (RAG) with LLMs to create actionable summary, and Alert Rule Refinement leverages multi-agent iterative feedbacks to improve alert rule quality. Evaluated on four real-world datasets from Company-X’s services, AlertGuardian significantly mitigates alert fatigue (94.8% alert reduction ratios) and accelerates fault diagnosis (90.5% diagnosis accuracy). Moreover, AlertGuardian improves 1,174 alert rules, with 375 accepted by SREs (32% acceptance rate). Finally, we share success stories and lessons learned about alert life-cycle management after the deployment of AlertGuardian in Company-X.
DOI 原文 ·
@inproceedings{paperbot9389,
title = {AlertGuardian: Intelligent Alert Life-Cycle Management for Large-scale Cloud Systems},
author = {Guangba Yu and Genting Mai and Rui Wang and Ruipeng Li and Pengfei Chen and Long Pan and Ruijie Xu},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00009}
}