一句话：提出 Graphectory 图编码智能体轨迹的时序与语义关系，实现以过程为中心的分析与在线监控，提升问题解决率。
智能体系统已成为一类新型软件系统，但其执行轨迹具有随机性与自适应性，现有评估多以最终成败为中心，无法解释智能体如何推理、规划与调整策略。作者受传统软件系统图结构化表示的启发，提出 Graphectory，用图系统化编码轨迹中的时序与语义关系，并据此设计以过程为中心的度量与分析。基于该表示，作者自动分析了 SWE-agent 与 OpenHands 两个主流智能体编程工作流、四种骨干大模型在 SWE-bench Verified 上的 4000 条轨迹（四分钟内完成），发现提示更丰富或模型更强时图结构更复杂、探索与验证更充分；已解决问题呈连贯的定位–打补丁–验证流程，未解决问题则出现混乱、重复与回溯；即便成功也常存在低效的冗长过程。作者进一步实现在执行中实时构建与分析 Graphectory/Langutory 的技术，检测到轨迹问题即向智能体发送诊断信息并在适用时回滚。实验表明，在线监控与过程分析配合适当干预，可将问题实例的解决率提升 6.9%–23.5%，并显著缩短轨迹且开销近乎为零。
Agentic systems are modern software systems: they consist of orchestrated modules, expose interfaces, and are deployed in software pipelines. Unlike conventional programs, their execution, i.e., trajectories, is inherently stochastic and adaptive to the problems they are solving. Evaluation of such systems is often outcome-centric, i.e., judging their performance based on success or failure at the final step . This narrow focus overlooks detailed insights about such systems, failing to explain how agents reason, plan, act, or change their strategies. Inspired by the structured representation of conventional software systems as graphs, we introduce Graphectory to systematically encode the temporal and semantic relations in such software systems. Graphectory facilitates the design of process-centric metrics and analyses to assess the quality of agentic workflows. Using Graphectory , we automatically analyze 4000 trajectories of two dominant agentic programming workflows, namely SWE-agent and OpenHands, with a combination of four backbone Large Language Models (LLMs), attempting to resolve SWE-bench Verified issues. Our fully automated analyses (completed within four minutes) reveal that: (1) agents using richer prompts or stronger LLMs exhibit more complex Graphectory , reflecting deeper exploration, broader context gathering, and more thorough validation before patch submission; (2) agents’ problem-solving strategies vary with both problem difficulty and the underlying LLM—for resolved issues, the strategies often follow coherent localization–patching–validation steps, while unresolved ones exhibit chaotic, repetitive, or backtracking behaviors; and (3) even when successful, agentic programming systems often display inefficient processes, leading to unnecessarily prolonged trajectories. We also implement a novel technique for real-time construction and analysis of Graphectory and Langutory during the agent’s execution to flag trajectory issues. Upon detecting such issues in the trajectory, the proposed technique notifies the agent with a diagnostic message and, when applicable, rolls back the trajectory. The experimental results show that online monitoring and process-centric analysis, when accompanied by appropriate interventions, can improve resolution rates by 6.9%-23.5% across models for problematic instances, while significantly shortening trajectories with near-zero overhead.
@article{paperbot3823,
title = {Process-Centric Analysis of Agentic Software Systems},
author = {Shuyang Liu and Yang Chen and Rahul Krishna and Saurabh Sinha and Jatin Ganhotra and Reyhaneh Jabbarvand},
journal = {Proceedings of the ACM on Programming Languages},
volume = {10},
number = {OOPSLA1},
year = {2026},
doi = {10.1145/3798271}
}