一句话：提出APPS-Failure-DB数据集与工具辅助定位框架，将LLM代码智能体的推理失败日志关联到代码故障。
论文针对LLM代码智能体失败常源于内部推理缺陷而非简单代码bug、但缺乏标注数据的问题，构建了APPS-Failure-DB：包含2450个来自Qwen 7B Coder在APPS任务上的经验证推理失败样本，并通过盲Oracle Agent（GPT-4o）提供真值标注。基于该数据集，作者提出工具辅助定位框架，结合覆盖追踪的动态分析与语义Tracer Agent构建可追踪映射，再由Debugger Agent完成推理阶段与代码行级定位。结果显示，73.82%的失败发生在执行阶段，推理阶段定位准确率为67.48%；但7B模型代码行级定位仍困难，F1仅0.0714。该工作同时贡献数据集与诊断框架。
Autonomous LLM-based code agents are rapidly advancing, yet their practical utility is hindered by a critical gap in our understanding of their failures. When an agent fails, the root cause is often not a simple code bug but a complex flaw in its internal reasoning process, which is captured in unstructured "hypothesis" logs. To date, there has been no annotated dataset to systematically study this phenomenon, especially on complex, competitive programming-style tasks. To address this gap, we introduce APPS-Failure-DB, a new benchmark dataset of 2450 unique, validated reasoning failures from an LLM agent (Qwen 7B Coder) tasked with solving problems from the APPS dataset. We generated this dataset using a novel "Oracle-Annotated Pipeline," where a blind Oracle Agent (GPT-4o) validates and provides ground-truth annotations for each logical failure. Building on this dataset, we propose and evaluate a new, tool-assisted localization framework that is the first to link the unstructured text of an agent’s hypothesis log to specific code-level faults. Our framework combines dynamic analysis (coverage tracing) with a semantic "Tracer Agent" to build a traceability map, enabling a final "Debugger Agent" to perform precise localization. Our results demonstrate that we can characterize failure patterns in agent reasoning, with 73.82% of failures occurring in the Execution-Phase, and achieve a reasoning-phase localization accuracy of 67.48%. While code-line-level localization remains challenging with our 7B parameter model (F1-Score of 0.0714), our framework establishes a foundation for future work with larger models. This work contributes both a new dataset for the research community and a novel framework for diagnosing agent reasoning failures.
@inproceedings{paperbot11784,
title = {A Localization Framework for Reasoning Faults in LLM-based Code Agents},
author = {Birudugadda Srivibhav and Kondam Sriman Reddy and Nakka Naga Bhuvith and Koleti Eswar Sai Ganesh and Shouvick Mondal},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3774748.3793627}
}