一句话：用随机输入下执行输出的一致性模式与 Dawid–Skene 模型估计生成代码的可靠性，为自校正代码生成提供更可信的反馈信号。
大语言模型在代码生成上进展显著，但在需要复杂算法或实现的任务上仍表现不佳。近期工作常以代码执行作为反馈，尤其是在从生成代码构造验证信号的自校正流程中；然而这些信号本身的可靠性未知，会带来误导性反馈、不必要的修改与错误答案。本文提出 ExeCRE，一个以执行一致性为指导的代码可靠性估计框架。它不依赖测试用例或 LLM 反馈来判定候选代码，而是在大量随机生成的输入上收集执行输出，将其投影为一致性信号，并用 Dawid–Skene 模型推断代码的潜在可靠性。作者将 ExeCRE 集成进代码生成的自校正流程。实验表明，ExeCRE 持续提升效果与稳定性，并大幅减少误导性修正信号：在 GPT-5.2 与 LiveCodeBench 上，对已正确代码产生误导性反馈的平均案例数从代表性自校正基线的 113.2 降到 14.0。作者还将其可靠性估计策略应用于基于代码的数学推理，观察到类似收益。
Large language models (LLMs) have made notable progress in code generation, but they still struggle on challenging tasks that require sophisticated algorithms or complex implementations. Recent methods increasingly use code execution as feedback, especially in self-correction pipelines that construct verification signals from generated code. However, these pipelines often depend on supervision signals whose reliability is unknown, which can introduce misleading feedback, unnecessary revisions, and incorrect final answers. To address this issue, we propose ExeCRE, an Execution-Consistency guided code Reliability Estimation framework. Instead of judging candidate code by tests or LLM feedback, ExeCRE estimates code reliability by statistically analyzing consistency patterns in execution outputs over a large number of randomly generated inputs. It collects execution outputs over generated inputs, projects them into consistency signals, and applies Dawid–Skene model to infer latent code reliability. We integrate ExeCRE into self-correction for code generation. Experiments show that ExeCRE consistently improves both effectiveness and stability, while substantially reducing misleading correction signals. Under GPT-5.2 on LiveCodeBench, the average number of misleading feedback cases on already correct code drops from 113.2 with a representative self-correction baseline to 14.0 with ExeCRE. As an additional study, we apply the same reliability estimation strategy to code-based mathematical reasoning and observe similar benefits. These results suggest that ExeCRE enables more reliable use of generated code in execution-based pipelines.
@inproceedings{paperbot12273,
title = {ExeCRE: Execution-Consistency Guided Reliability Estimation for Self-Correcting Code Generation},
author = {Yiru Dong and Richong Zhang and Fanshuang Kong and Si Chen},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3837445}
}