一句话：提出从推理层级、中间推理过程质量与真实项目语境三方面全面评估 LLM 代码推理能力的研究方向。
代码推理已成为评估大语言模型（LLM）编程任务表现的新范式。然而，现有技术多以显式方式考察 LLM 的代码推理能力（例如输入/输出预测），对于 LLM 能否在需要模拟代码执行的隐式任务中整合执行推理，仍缺乏探索。此外，当前评估策略过于简化，既限制了研究者对 LLM 推理过程（而非仅最终预测）的分析，也妨碍了对其在真实项目中泛化能力的严格评估。本论文为研究提案，强调需要对 LLM 的代码推理能力进行更全面、更贴近真实的评估，并提出三个研究方向：（1）用不同层级的代码推理任务挑战 LLM；（2）评估 LLM 中间推理过程的质量；（3）在真实世界语境下评估 LLM 的代码推理表现。作为提案，本文尚未给出具体实验结果。
Code reasoning has emerged as a new paradigm for evaluating LLMs’ performance on programming tasks. While existing techniques assess LLMs’ code reasoning abilities in an explicit manner—typically through tasks such as input/output prediction—the extent to which LLMs can implicitly integrate execution reasoning into tasks that require simulating the code execution remains largely unexplored. Moreover, the oversimplified nature of current evaluation strategies limits researchers’ ability to analyze the reasoning process of LLMs beyond their final predictions, and hinders a rigorous assessment of their generalizability to real-world projects. In this research proposal, we emphasize the need for a comprehensive and realistic evaluation of LLMs’ code reasoning capabilities. To address the limitations of prior work, we outline three research directions: (1) challenging LLMs with code reasoning tasks at different levels, (2) assessing the quality of LLMs’ intermediate reasoning processes, and (3) evaluating LLMs’ code reasoning performance under real-world contexts.