一句话：用差分模糊测试检验 LLM 代码重构的功能等价性，发现 19–35% 的重构会改变程序语义。
随着大语言模型（LLM）被广泛用于自动化代码重构，如何确保 LLM 生成的重构与原始实现功能等价变得至关重要。现有工作通常依赖预定义测试用例来评估正确性，本文则采用差分模糊测试来检查 LLM 生成重构的功能等价性：该方法无需预定义测试用例，而是自动生成并执行成千上万个测试输入，比较原程序与重构程序的行为，从而探索更大的输入空间。作者对六种 LLM（CodeLlama、Codestral、Star-Chat2、Qwen-2.5、Olmo-3 和 GPT-4o）在三个数据集、两类重构任务上进行了大规模评估。结果表明，LLM 存在不可忽视的改动程序语义倾向，产生了 19–35% 功能不等价的重构；其中约 21% 的不等价重构未被所评估的三个数据集既有测试套件检测到。这些发现说明，依赖既有测试可能高估 LLM 生成代码重构的功能等价性，此类重构仍容易出现语义偏差。
With the rapid adoption of large language models (LLMs) in automated code refactoring, assessing and ensuring functional equivalence between LLM-generated refactoring and the original implementation becomes critical. While prior work typically relies on predefined test cases to evaluate correctness, in this work, we leverage differential fuzzing to check functional equivalence in LLM-generated code refactorings. Unlike test-based evaluation, a differential fuzzing-based equivalence checker needs no predefined test cases and can explore a much larger input space by executing and comparing thousands of automatically generated test inputs. In a large-scale evaluation of six LLMs (CodeLlama, Codestral, Star-Chat2, Qwen-2.5, Olmo-3, and GPT-4o) across three datasets and two refactoring types, we find that LLMs show a non-trivial tendency to alter program semantics, producing 19–35% functionally non-equivalent refactorings. Our experiments further demonstrate that about 21% of these non-equivalent refactorings remain undetected by the existing test suites of the three evaluated datasets. Collectively, the findings of this study imply that reliance on existing tests might overestimate functional equivalence in LLM-generated code refactorings, which remain prone to semantic divergence.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2602-15761,
title = {A Differential Fuzzing-Based Evaluation of Functional Equivalence in LLM-Generated Code Refactorings},
author = {Simantika Bhattacharjee Dristi and Matthew B. Dwyer},
booktitle = {FSE},
year = {2026},
doi = {10.1145/3803437.3806717}
}