一句话：通过对比正确与错误代码的隐藏状态，发现并利用 LLM 内部的代码正确性表示来筛选更可靠的生成代码。
尽管大语言模型（LLM）在代码生成上表现出色，但输出概率常与代码正确性弱相关，且仅反映生成过程的最终结果。受 LLM 内部编码真实性等概念的启发，本文探究 LLM 是否同样表示代码正确性。作者对同一编程任务的正确与错误代码对，对比其隐藏状态，从而识别出 LLM 内部的正确性表示。在四个 LLM 上的实验表明，利用这种提取出的正确性表示进行排序，优于标准对数似然排序以及模型口头表达的信心度。进一步，作者探索如何在不执行测试的情况下，用该内部正确性信号选择更高质量的代码样本。该工作展示了利用内部表示提升代码生成系统与 LLM 可靠性的潜力。
Despite the effectiveness of large language models (LLMs) for code generation, they often output incorrect code. One reason is that model output probabilities are often not well-correlated with correctness, and reflect only the final output of the generation process. Inspired by findings that LLMs internally encode concepts like truthfulness, this paper explores if LLMs similarly represent code correctness. Specifically, we identify a correctness representation inside LLMs by contrasting the hidden states between pairs of correct and incorrect code for the same programming tasks. By experimenting on four LLMs, we show that exploiting this extracted correctness representation outperforms standard log-likelihood ranking, as well as verbalized model confidence. Furthermore, we explore how this internal correctness signal can be used to select higher-quality code samples, without requiring test execution. Ultimately, this work demonstrates how leveraging internal representations can enhance code generation systems and make LLMs more reliable, thus improving confidence in automatically generated code.