一句话：提出自动收集框架 REEF，从开源仓库构建多语言高质量漏洞-修复对数据集并生成漏洞解释。
软件漏洞危害严重，近年来数据驱动的自动程序修复取得进展，但现有基准存在语言单一、规模小、过时、代码库单一以及漏洞解释质量低等问题。本文提出自动收集框架 REEF，从开源仓库采集真实世界漏洞及其修复。作者开发多语言爬虫收集漏洞与修复，设计度量指标筛选高质量漏洞-修复对，并提出基于神经语言模型的方法生成高质量漏洞解释，从而产出信息丰富的修复信息。实验表明，REEF 能收集高质量漏洞-修复对并生成强解释。所构建数据集包含 7 种编程语言中的 4,466 个 CVE 和 30,987 个补丁（涵盖 236 个 CWE）及详细相关信息，在规模、覆盖面和数据质量上均优于现有基准；人类专家评估进一步确认其漏洞解释质量较高。
Software plays a crucial role in our daily lives, and therefore the quality and security of software systems have become increasingly important. However, vulnerabilities in software still pose a significant threat, as they can have serious consequences. Recent advances in automated program repair have sought to automatically detect and fix bugs using data-driven techniques. Sophisticated deep learning methods have been applied to this area and have achieved promising results. However, existing benchmarks for training and evaluating these techniques remain limited, as they tend to focus on a single programming language and have relatively small datasets. Moreover, many benchmarks tend to be outdated and lack diversity, focusing on a specific codebase. Worse still, the quality of bug explanations in existing datasets is low, as they typically use imprecise and uninformative commit messages as explanations. To address these issues, we propose an automated collecting framework REEF to collect REal-world vulnErabilities and Fixes from open-source repositories. We focus on vulnerabilities since they are exploitable and have serious consequences. We develop a multi-language crawler to collect vulnerabilities and their fixes, and design metrics to filter for high-quality vulnerability-fix pairs. Furthermore, we propose a neural language model-based approach to generate high-quality vulnerability explanations, which is key to producing informative fix messages. Through extensive experiments, we demonstrate that our approach can collect high-quality vulnerability-fix pairs and generate strong explanations. The dataset we collect contains 4,466 CVEs with 30,987 patches (including 236 CWE) across 7 programming languages with detailed related information, which is superior to existing benchmarks in scale, coverage, and quality. Evaluations by human experts further confirm that our framework produces high-quality vulnerability explanations.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{WangLPGCWGL23,
title = {REEF: A Framework for Collecting Real-World Vulnerabilities and Fixes},
author = {Chaozheng Wang and Zongjie Li and Yun Pena and Shuzheng Gao and Sirong Chen and Shuai Wang and Cuiyun Gao and Michael R. Lyu},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00199}
}