一句话：将歧义自然语言描述的修复分解为解释分布的测试与程序修复、以及对比式规约推断，提升LLM代码生成正确率。
自然语言问题描述中的歧义会使 LLM 生成错误程序，而直接提示 LLM 澄清歧义常产生无关或不一致的修改。本文提出歧义问题描述的自动修复，将任务分解为两步：先用传统测试与程序修复技术分析并修复 LLM 对描述的解释（由所诱导的程序分布刻画）；再通过对比式规约推断，依据分布变化精炼描述。作者实现工具 SPEC-FIX，在 GPT-4o、DeepSeek-V3 等四个 LLM 与 HumanEval+、MBPP+、LiveCodeBench 三个基准上评估。无需人工干预或外部信息，SPEC-FIX 修改了 43.58% 的描述，被修改集合上 Pass@1 提升 30.9%，全基准绝对提升 4.09%；修复还可跨模型迁移，使其他模型性能提升 10.48%。
The growing use of large language models (LLMs) has increased the importance of natural language (NL) in software engineering. However, ambiguity of NL can harm software quality, as unclear problem descriptions may lead to incorrect program generation. Detecting and resolving such ambiguity is challenging, motivating our introduction of the automated repair of ambiguous NL descriptions, which we approach by reducing code generation uncertainty and better aligning NL with input–output examples. Ambiguity repair is difficult for LLMs because they must understand how their interpretation of a description changes when the text is altered. We find that directly prompting LLMs to clarify ambiguity often produces irrelevant or inconsistent edits. To address this, we decompose this task into two simpler steps: (1) analyzing and repairing the LLM’s interpretation of the description — captured by the distribution of programs it induces — using traditional testing and program repair, and (2) refining the description based on distribution changes via a method we call contrastive specification inference. We implement this approach in a tool called SPEC-FIX and evaluate it using four state-of-the-art LLMs (GPT-4o, GPT-4o-mini, DeepSeek-V3, and Qwen2.5-Coder-32B-Instruct) on three popular code generation benchmarks (HumanEval+, MBPP+ and LiveCodeBench). Without human intervention or external information, SPECFIX modified 43.58% of descriptions, improving Pass@1 on the modified set by 30.9%. This yields a 4.09% absolute improvement across the entire benchmark. Repairs also transfer across models: descriptions repaired for one model improve other models’ performance by 10.48%.
DOI 原文 ·
@inproceedings{paperbot9402,
title = {Automated Repair of Ambiguous Problem Descriptions for LLM-Based Code Generation},
author = {Haoxiang Jia and Robbie Morris and He Ye and Federica Sarro and Sergey Mechtaev},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00038}
}