一句话：提出 PatUntrack，利用 LLM 从无追踪不安全代码的漏洞 issue 报告自动生成 Top-K 不安全代码与补丁示例对。
安全补丁对开源项目稳定性至关重要，但漏洞披露前完成修补十分困难。实践中，安全人员通常追踪漏洞 issue 报告并分析相关不安全代码来生成补丁；然而，当不安全代码未被显式指定且无法在仓库中追踪时，补丁生成会受到严重限制。本文提出 PatUntrack，一种从无追踪不安全代码的 issue 报告中自动生成补丁示例的方法。该方法利用 auto-prompting 优化大语言模型：首先从漏洞 issue 报告生成漏洞触发路径（VTP）的完整描述，然后借助外部 golden knowledge 纠正 VTP 描述中的潜在幻觉，最后基于纠正后的 VTP 生成 Top-K 不安全代码与补丁示例对。作者在 5,465 个漏洞 issue 报告上评估，结果表明 PatUntrack 取得最佳性能，较传统 LLM 基线平均提升 MatchFix +17.7%、Fix@10 +14.6%。此外，该方法应用于 76 个新披露漏洞 issue 报告，37 条来自报告作者的回复中有 27 条确认生成补丁示例有用。
Security patches are essential for enhancing the stability and robustness of projects in the open-source software community. While vulnerabilities are officially expected to be patched before being disclosed, patching vulnerabilities is complicated and remains a struggle for many organizations. To patch vulnerabilities, security practitioners typically track vulnerable issue reports (IRs), and analyze their relevant insecure code to generate potential patches. However, the relevant insecure code may not be explicitly specified and practitioners cannot track the insecure code in the repositories, thus limiting their ability to generate patches. In such cases, providing examples of insecure code and the corresponding patches would benefit the security developers to better locate and resolve the actual insecure code. In this paper, we propose PatUntrack, an automated approach to generating patch examples from IRs without tracked insecure code. PatUntrack utilizes auto-prompting to optimize the Large Language Model (LLM) to make it applicable for analyzing the vulnerabilities described in IRs and generating appropriate patch examples. Specifically, it first generates the completed description of the Vulnerability-Triggering Path (VTP) from vulnerable IRs. Then, it corrects potential hallucinations in the VTP description with external golden knowledge. Finally, it generates Top-K pairs of Insecure Code and Patch Example based on the corrected VTP description. To evaluate the performance of PatUntrack, we conducted experiments on 5,465 vulnerable IRs. The experimental results show that PatUntrack can obtain the highest performance and improve the traditional LLM baselines by +17.7% (MatchFix) and +14.6% (Fix@10) on average in patch example generation. Furthermore, PatUntrack was applied to generate patch examples for 76 newly disclosed vulnerable IRs. 27 out of 37 replies from the authors of these IRs confirmed the usefulness of the patch examples generated by PatUntrack, indicating that they can benefit from these examples for patching the vulnerabilities.
@inproceedings{paperbot8315,
title = {PatUntrack: Automated Generating Patch Examples for Issue Reports without Tracked Insecure Code},
author = {Ziyou Jiang and Lin Shi and Guowei Yang and Qing Wang},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3694982}
}