一句话：提出 ExeRScope 工具与启发式，深入分析 LLM 代码执行推理结果并理解代码属性的影响。
代码执行推理正成为评估大语言模型（LLM）编程任务能力的非功能指标。现有框架与基准（如 CruxEval）通常只关注 LLM 在有限程序上预测给定代码的输入/输出或中间变量状态/值，缺乏对结果进行更深入分析的工具。缺少此类工具，关于 LLM 代码执行推理的观察无法泛化到更多数据集，阻碍研究界与从业者设计下一代具有更好代码执行推理能力的 LLM。本文提出 ExeRScope，它是一系列工具与启发式方法，用于分析代码执行推理框架的结果，以理解代码属性在被研究基准中的影响。借助该工具，分析可以泛化到具有相似属性的代码，而无需迫切地设计更多基准，从而避免这一繁琐工作。
Code Executing Reasoning is becoming a new non-functional metric that assesses the ability of large language models (LLMs) in programming tasks. State-of-the-art frameworks and benchmarks (CruxEval) usually focus on LLM's prediction of a given code's input/output or intermediate variable states/values on limited programs. However, there is no tool for more in-depth analysis of the results. Without such a tool, the observations about LLM's code execution reasoning cannot be generalized to more datasets, preventing the research community and practitioners from devising the next generation of LLMs with better code execution reasoning abilities. This paper introduces ExeRScope, a series of tools and heuristics to analyze the result of code execution reasoning frameworks to understand the impact of code properties in the studied benchmarks. With such tooling, analysis can be generalized to code with similar properties without the urgent need to design more benchmarks, which is a cumbersome effort.