一句话：提出提示复述测试，通过改写提示生成多版代码并交叉验证语义一致性，以检测LLM生成代码缺陷。
LLM已被广泛用于代码生成，但生成代码的质量仍存疑，自动验证仍是难题。本文提出“提示复述测试”：观察到正确代码片段之间通常具有内在一致性，而有缺陷代码之间未必如此，因此可通过检测不一致性发现缺陷。方法是将给定提示改写为多个语义等价的提示，让LLM生成多版代码，再通过交叉验证检查这些代码之间的语义关系是否仍然成立。作者在HumanEval上进行初步评估，使用GPT-4o生成的程序，能检测出70.6%的错误程序，假阳性率为6.1%。
Large Language Models (LLMs) have been widely used for code generation. However, the quality of generated code is still questionable, and code validation remains a challenging problem. In this paper, we propose a novel solution called paraphrase prompt testing. Our intuitive observation is that intrinsic consistency always exists among correct code pieces but may not exist among flawed code pieces, so we can detect flaws in the code by detecting inconsistencies. Therefore, we can vary a given prompt to multiple prompts with paraphrasing, and to ask the LLM to acquire multiple versions of generated code, so that we can validate whether the semantic relations still hold in the acquired code through cross-validation. Our initial evaluation on HumanEval shows that paraphrase prompt testing is able to detect 70.6% of the erroneous programs generated by GPT-4o, with a false positive rate of 6.1%.