一句话：提出多语言基准 FuseApplyBench，系统评估 LLM 在代码编辑应用中的三类错误，并给出经微调的 FuseApply 模型。
随着语言模型和大语言模型的兴起，代码编辑受到关注：通常由大模型生成代码修改草稿，再由较小模型完成代码编辑应用（Code Editing Apply, CEA）。然而 CEA 容易出错，现有基准未系统评估大模型处理这些问题的能力。本文提出 FuseApplyBench，该基准围绕 CEA 任务中的三类主要错误类型评估模型表现。作者在 FuseApplyBench 流程之上收集数据集进行微调，得到 FuseApply，以提升代码修改应用的可靠性。实验将 FuseApply、四个广泛使用的开源大模型以及 Kortix-FastApply 在 FuseApplyBench 上进行对比。结果显示，FuseApply 在可信性与准确性指标上显著提升，而其他模型表现较弱，说明提升大模型在代码编辑应用方面的能力仍有很大空间。
With the rise of Language Models (LMs) and Large Language Models (LLMs), their potential for code editing (CE) has gained attention. An approach is to have LLMs generate draft code modifications, which are then refined by smaller LMs in further Code Editing Apply (CEA). However, CEA is error-prone, and existing benchmarks do not systematically evaluate LLM performance in handling these issues. We introduce FuseApplyBench, a benchmark designed to evaluate LLM performance across three major error types in CEA tasks. Atop FuseApplyBench's pipeline, we collect datasets to perform fine-tuning, enhancing code modifications' reliability (denoted as FuseApply). We benchmark FuseApply, four widely used open source LLMs, and Kortix-FastApply on FuseApplyBench. Results show that FuseApply significantly improves trustworthiness and accuracy metrics, while other models demonstrate weaker performance, highlighting opportunities for advancing LLM in CE.
@inproceedings{paperbot9599,
title = {FuseApplyBench: Multilingual Benchmark for Trustworthy Code Edit Applying Task},
author = {Ming Liang and Qingyu Zhang and Zhipeng Zuo and Shaoqiang Zheng and Dajun Chen and Wei Jiang and Yong Li},
booktitle = {ISSTA},
year = {2025},
doi = {10.1145/3713081.3732929}
}