一句话:结合静态分析、随机测试与大语言模型,为可能过拟合的补丁生成并排序能揭示缺陷的测试用例。
补丁正确性评估的目标是识别过拟合补丁,即能让所有测试通过、但并未真正修复缺陷的补丁。已有自动化技术大多只给出正确/不正确的二分类结果,不解释判定依据。本文以工具演示形式提出 FixCheck,它融合静态分析、随机测试与大语言模型(LLM)来改进补丁正确性分析:首先用静态分析和随机测试生成一批与原始失败测试用例相似的测试用例,然后借助 LLM 为每个新测试用例推导有意义的断言,最后执行这些测试,将失败的测试选出并依据其揭示补丁缺陷的可能性进行优先级排序,从而为可能不正确的补丁提供能暴露错误的测试,提升分析过程的可解释性与效果。
Patch correctness assessment aims at effectively detecting overfitted patches, i.e., patches that causes all tests to pass but do not actually fix the bug. Although several automated techniques for assessing patch correctness have been proposed, these techniques typically yield a binary result (correct/incorrect) without providing any additional information explaining the rationale behind the decision of classifying a patch as correct or incorrect. This tool demo paper presents FixCheck, a tool based on static analysis, random testing and Large Language Models (LLMs), that seeks to improve the patch correctness analysis process by providing fault-revealing tests for potentially incorrect patches. To this end, FixCheck first employs static analysis and random testing to generate a comprehensive set of test cases that are similar to the original failing test case. Then, FixCheck relies on LLMs to derive meaningful assertions for each new test case. Finally, FixCheck executes the generated tests, and those that fail are selected and prioritized based on their likelihood of revealing a defect in the patch.