一句话：提出 FLAME，用 LLM 引导标注与多模型加权投票实现编程作业细粒度可解释故障定位。
该论文关注教育场景下编程作业的自动故障定位问题。现有方法多停留在方法级且缺乏解释，难以给学生可操作的反馈；部分行级方法又依赖直接预测行号，不适合 LLM。为此，作者提出 FLAME：利用编程作业特有的丰富上下文引导 LLM 标注故障代码行并生成解释，而不是直接输出行号；同时引入加权多模型投票策略，聚合多个 LLM 的结果以判断每行代码的可疑度，提高定位可靠性。实验表明，FLAME 在编程作业上优于最先进的故障定位基线，在 top-1 上比最佳基线多定位 207 个故障；在通用软件代码库 Defects4J 上也展现出良好泛化能力，并优于所有基线。
Providing timely and personalized guidance for students’ programming assignments, particularly by indicating fine-grained error locations with explanations, offers significant practical value for helping students complete assignments and enhance their learning outcomes. In recent years, various automated Fault Localization (FL) techniques, particularly those leveraging Large Language Models (LLMs), have demonstrated promising results in identifying errors in programs. However, existing fault localization techniques face challenges when applied to educational contexts. Most approaches operate at the method level without explanatory feedback, resulting in granularity too coarse for students who need actionable insights to identify and fix their errors. While some approaches attempt line-level fault localization, they often depend on predicting line numbers directly in numerical form, which is ill-suited to LLMs. To address these challenges, we propose FLAME, a fine-grained, explainable Fault Localization method tailored for programming assignments via LLM-guided Annotation and Model Ensemble. FLAME leverages rich contextual information specific to programming assignments to guide LLMs in identifying faulty code lines. Instead of directly predicting line numbers, we prompt the LLM to annotate faulty code lines with detailed explanations, enhancing both localization accuracy and educational value. To further improve reliability, we introduce a weighted multi-model voting strategy that aggregates results from multiple LLMs to determine the suspiciousness of each code line. Extensive experimental results demonstrate that FLAME outperforms state-of-the-art fault localization baselines on programming assignments, successfully localizing 207 more faults at top-1 over the best-performing baseline. Beyond educational contexts, FLAME also generalizes effectively to general-purpose software codebases, outperforming all baselines on the Defects4J benchmark.
DOI 原文 ·
@inproceedings{paperbot9465,
title = {Explainable Fault Localization for Programming Assignments via LLM-Guided Annotation},
author = {Fang Liu and Tianze Wang and Li Zhang and Zheyu Yang and Jing Jiang and Zian Sun},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00167}
}