paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation

通过语义三角测量减少 LLM 生成代码中的幻觉
OOPSLA 10(OOPSLA2)2026匹配度 3/10 · 论文关注 LLM 生成代码的幻觉检测与正确性提升,属于以 LLM 本身为研究目标而非将其作为工具的研究,与用户核心方向(PL→AI 静态分析、AI→PL 改进静态分析/形式化验证、浮点/贝叶斯程序分析)交集有限。虽涉及程序正确性验证的邻域概念,但未触及静态分析、形式化验证或浮点/贝叶斯分析等具体技术。引用 3 inbox
Yihan Dai, Sijie Liang, Haotian Xu, Peichu Xie, Sergey Mechtaev
crossref: 2026-10-02

AI 速览

一句话:提出语义三角化,将问题变换为需不同算法的离解变体并检查独立解一致性,以降低 LLM 代码幻觉。

LLM 生成代码常因幻觉含 bug;无形式规约时,用 LLM 生成的测试或自动形式化规约等代理评估正确性并不可靠,因为同一模型的错误相关,代理常相互印证而非发现错误。本文提出语义三角化:将原问题变换为可能需要根本不同算法的离解变体,并检查两问题独立采样解的一致性,以去相关模型错误。作者给出理论要求,并在形式化幻觉模型下证明其提高正确性置信度;基于问题反转、分解和解枚举实例化四种方法。在 LiveCodeBench 和 CodeElo 上评估 GPT-4o、DeepSeek-V3、Gemini 2.5 Flash,选择正确程序概率较基线提高 16%,选择或弃权场景下可靠性与 F1 各提高 7%。

问题
LLM 生成代码常因幻觉包含 bug,而缺少形式规约时,测试或自动形式化规约等 LLM 生成的代理因错误相关,往往无法可靠发现错误。
方法
提出语义三角化,将原问题变换为可能需要不同算法的离解变体,并检查独立采样解的一致性;给出理论要求与形式化证明,并实例化问题反转、分解、解枚举四种方法。
评估
在 LiveCodeBench 与 CodeElo 上,对 GPT-4o、DeepSeek-V3、Gemini 2.5 Flash,与测试生成、蜕变测试、自动形式化规约基线比较,选择正确程序概率提高 16%,选择或弃权场景下可靠性与 F1 各提高 7%。
原文摘要(Abstract)

Large language models (LLMs) can generate executable code from natural language descriptions, but the resulting programs frequently contain bugs due to hallucinations. In the absence of formal specifications, existing approaches attempt to assess correctness using LLM-generated proxies such as tests or auto-formalized specifications. However, these proxies are produced by the same imperfect models and thus often corroborate rather than catch errors, especially when the model exhibits correlated errors. We introduce semantic triangulation, a theory-grounded framework that decorrelates model errors by transforming the original problem into a dissociative variant---one likely requiring a fundamentally different algorithm---and checks consistency between independently sampled solutions to both problems. We identify theoretical requirements for this framework, and we prove that under a formal model of LLM hallucinations, these properties confer higher confidence in program correctness. We instantiate the framework through four concrete triangulation methods based on problem inversion, decomposition, and solution enumeration. Evaluated on LiveCodeBench and CodeElo across GPT-4o, DeepSeek-V3, and Gemini 2.5 Flash, our tool increases the probability of selecting a correct program by 16% over baselines (test generation, metamorphic testing, and auto-formalized specifications) and achieves 7% higher reliability and 7% higher F1 score in selection-or-abstention scenarios, while being the only method that consistently handles inexact problems admitting multiple valid solutions.

链接与引用

DOI 原文 · arXiv · PDF(开放获取) · DBLP

BibTeX
@article{abs-2511-12288,
  title = {Reducing Hallucinations in LLM-Generated Code via Semantic Triangulation},
  author = {Yihan Dai and Sijie Liang and Haotian Xu and Peichu Xie and Sergey Mechtaev},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {10},
  number = {OOPSLA2},
  year = {2026},
  doi = {10.1145/3839525}
}