paperbot · PL 论文追踪

全部论文 待读 收藏

P1 Hierarchical Metamorphic Test Oracles for Agent Evaluation

面向智能体评估的分层蜕变测试预言
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 5/10 · 论文提出面向 Agent 的分层蜕变测试预言,属于 Agentic 程序的测试与调试,与用户对 Agentic 程序分析/验证的兴趣有交叉;但核心是动态测试而非静态分析,也未涉及 LLM/Agent 改进静态分析、浮点或贝叶斯程序分析,因此匹配度中等。引用 0 inbox
Demetrius Hernandez
crossref: 2026-09-12

AI 速览

一句话:提出 MESA 框架,用目标、策略、动作三层蜕变测试预言评估智能体,并按层级分解失败。

随着自主系统从数据驱动组件转向能推理、规划和行动的意图驱动智能体,传统测试方法变得不足。智能体产生执行轨迹,其正确性依赖目标、策略和动作构成的层级意图。本文提出 MESA(Metamorphic Evaluation of Systems and Agents),一个模块化框架,将蜕变测试原则适配到智能体评估,通过目标、策略和动作三层预言检查跨抽象层级的一致性关系。与二元成功指标不同,MESA 可按层级分解失败,支持定向调试。该工作将蜕变关系确立为智能体测试的原则性方法,并提供理论基础与实用工具。

问题
传统测试难以评估从数据驱动组件转向意图驱动智能体的系统,其执行轨迹的正确性依赖目标、策略和动作构成的层级意图。
方法
MESA 将蜕变测试原则扩展为目标、策略、动作三层预言,检查跨抽象层级的一致性关系,并按层级分解失败以支持定向调试。
评估
摘要未报告定量实验,作者主要通过框架设计、理论基础与实用工具构建来论证该方法的可行性。
原文摘要(Abstract)

As autonomous systems transition from data-driven components to intent-driven agents that reason, plan, and act, traditional testing approaches become inadequate. Agents produce execution traces, and correctness depends on hierarchical intent spanning goals, strategies, and actions. We present MESA (Metamorphic Evaluation of Systems and Agents), a modular framework that adapts metamorphic testing principles to agent evaluation through three oracle layers (Goal, Strategy, and Action) that verify consistency relations across abstraction levels. Unlike binary success metrics, MESA decomposes failures by level, enabling targeted debugging. Our framework establishes metamorphic relations as a principled approach to agent testing, providing theoretical foundations and practical tooling for testing modern agentic systems.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot11530,
  title = {Hierarchical Metamorphic Test Oracles for Agent Evaluation},
  author = {Demetrius Hernandez},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3774748.3787754}
}