paperbot · PL 论文追踪

全部论文 待读 收藏

Beyond Outcomes: Toward Process-Aware Software Agents

超越结果:迈向过程感知的软件智能体
ASE Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering2026引用 0 inbox
Shuyang Liu
crossref: 2026-10-09

AI 速览

一句话:提出以执行轨迹为中心的过程感知框架,用 Graphectory 和 Langutory 刻画编程智能体的行为模式与计划遵循度。

自主编程智能体通常仅以是否解决软件任务来评估,这无法揭示导致成败的执行过程。本文提出以过程为中心的轨迹评估方法:Graphectory 捕捉智能体执行的结构组织,Langutory 将底层动作抽象为软件工程阶段以刻画问题解决策略,并通过计划遵循度将观察到的执行与预期工作流关联。在数千条轨迹上的分析揭示了仅靠结果无法解释的行为模式,包括成功运行中持续存在的低效,以及智能体将计划转化为执行时的显著差异。这些发现凸显理解智能体执行过程的重要性,为软件智能体的过程感知优化奠定基础。

问题
现有编程智能体评估主要关注任务是否成功,缺乏对导致成功或失败的执行过程的深入理解。
方法
提出过程中心的轨迹分析框架,包括捕捉执行结构组织的 Graphectory、将底层动作抽象为软件工程阶段的 Langutory,以及关联观察执行与预期工作流的计划遵循度。
评估
在数千条智能体执行轨迹上进行实证分析,发现成功运行中的持续低效和计划到执行转化中的显著差异等结果无法解释的行为模式。
原文摘要(Abstract)

Autonomous programming agents are commonly evaluated by whether they solve software tasks, providing limited insight into the execution processes that lead to success or failure. We present a process-centric approach to evaluating agent behavior through trajectories. We introduce Graphectory, which captures the structural organization of agent execution, and Langutory, which abstracts low-level actions into software engineering phases to characterize problem-solving strategies. We further relate observed execution to intended workflows through plan compliance. Across thousands of trajectories, our analysis uncovers behavioral patterns that outcomes alone cannot explain, including persistent inefficiencies in successful runs and substantial variation in how agents translate plans into execution. These findings highlight the importance of understanding how agents execute, laying the foundation for the process-aware optimization of software agents.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot12522,
  title = {Beyond Outcomes: Toward Process-Aware Software Agents},
  author = {Shuyang Liu},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3844569}
}