paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Freezing the Crime Scene: A State Snapshot Paradigm for Reproducible Agentic SRE Evaluation

冻结犯罪现场:一种面向可复现 Agentic SRE 评估的状态快照范式
FSE Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering2026匹配度 4/10 · 论文关注 Agentic SRE 的可复现评测与基准基础设施,与用户对 Agentic 程序和 benchmark 的兴趣有间接交集;但它不涉及静态分析、形式化验证、浮点或贝叶斯程序分析,核心是 LLM Agent 的运维评测,与研究方向匹配有限。引用 0 inbox
Guangba Yu, Yilun Wang, Michael R. Lyu
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:提出状态快照范式,将运维状态与运行时执行解耦,以确定性数字孪生实现可复现的 Agentic SRE 过程评估。

LLM 正在改变站点可靠性工程(SRE),但现有评估框架难以兼顾生态效度与可复现性:真实集群基准受环境随机性影响,静态数据集又缺乏智能体推理所需的交互性。论文提出“状态快照”范式,将运维状态与运行时执行解耦,构造确定性的数字孪生,从而在保留真实运维复杂度的同时消除执行随机性。该基础设施支持以过程为中心的严格评估,能够揭示仅依赖结果指标所忽视的关键现象。作者认为这一范式不仅是基准,还可作为进化引擎,用于训练下一代可信的 Agentic SRE。

问题
现有 LLM/Agentic SRE 评估在真实集群的生态效度与实验可复现性之间存在内在张力,静态数据集也无法支持智能体交互推理。
方法
提出状态快照范式,把运维状态从运行时执行中解耦,用确定性数字孪生冻结现场,支持过程中心的可复现评估。
评估
通过该范式开展过程中心评估,揭示结果指标遗漏的关键洞察,并定位为训练可信 Agentic SRE 的进化引擎;摘要未报告具体实验数据。
原文摘要(Abstract)

While Large Language Models (LLMs) are transforming Site Reliability Engineering (SRE), current evaluation frameworks face an inherent tension between ecological validity and reproducibility. Live cluster benchmarks suffer from environmental stochasticity, while static datasets lack the interactivity required for agentic reasoning. To resolve this, we propose the state snapshot paradigm, a methodological shift that decouples operational state from runtime execution to create deterministic digital twins. This infrastructure enables rigorous process-centric evaluation, revealing critical insights that outcome-based metrics overlook. Ultimately, this paradigm serves not merely as a benchmark, but as an evolutionary engine for training the next generation of trustworthy Agentic SREs.

链接与引用

DOI 原文 · PDF(开放获取) · DBLP

BibTeX
@inproceedings{YuWL26,
  title = {Freezing the Crime Scene: A State Snapshot Paradigm for Reproducible Agentic SRE Evaluation},
  author = {Guangba Yu and Yilun Wang and Michael R. Lyu},
  booktitle = {FSE},
  year = {2026},
  doi = {10.1145/3803437.3806092}
}