一句话:提出状态快照范式,将运维状态与运行时执行解耦,以确定性数字孪生实现可复现的 Agentic SRE 过程评估。
LLM 正在改变站点可靠性工程(SRE),但现有评估框架难以兼顾生态效度与可复现性:真实集群基准受环境随机性影响,静态数据集又缺乏智能体推理所需的交互性。论文提出“状态快照”范式,将运维状态与运行时执行解耦,构造确定性的数字孪生,从而在保留真实运维复杂度的同时消除执行随机性。该基础设施支持以过程为中心的严格评估,能够揭示仅依赖结果指标所忽视的关键现象。作者认为这一范式不仅是基准,还可作为进化引擎,用于训练下一代可信的 Agentic SRE。
While Large Language Models (LLMs) are transforming Site Reliability Engineering (SRE), current evaluation frameworks face an inherent tension between ecological validity and reproducibility. Live cluster benchmarks suffer from environmental stochasticity, while static datasets lack the interactivity required for agentic reasoning. To resolve this, we propose the state snapshot paradigm, a methodological shift that decouples operational state from runtime execution to create deterministic digital twins. This infrastructure enables rigorous process-centric evaluation, revealing critical insights that outcome-based metrics overlook. Ultimately, this paradigm serves not merely as a benchmark, but as an evolutionary engine for training the next generation of trustworthy Agentic SREs.