一句话：提出控制依赖预测与数据依赖预测两种预训练目标，提升漏洞分析并可直接进行依赖分析。
漏洞分析对软件安全至关重要。现有代码预训练目标要么忽略控制/数据依赖等语义结构，要么只学习使用该结构，尚未探究学习分析语义结构知识能否带来收益。本文提出两种新颖预训练目标：控制依赖预测（CDP）与数据依赖预测（DDP），分别基于源代码预测语句级控制依赖和 token 级数据依赖。CDP/DDP 引导模型学习细粒度依赖分析知识，预训练后既可微调增强漏洞代码理解，也可直接用于部分与完整函数的依赖分析。作者以 Transformer 模型 PDBERT 实现，并在漏洞检测、分类、评估三项任务及程序依赖分析上评估；结果显示 PDBERT 达到当前最优，控制/数据依赖预测 F1 分别超过 99% 和 94%。
Vulnerability analysis is crucial for software security. Inspired by the success of pre-trained models on software engineering tasks, this work focuses on using pre-training techniques to enhance the understanding of vulnerable code and boost vulnerability analysis. The code understanding ability of a pre-trained model is highly related to its pre-training objectives. The semantic structure, e.g., control and data dependencies, of code is important for vulnerability analysis. However, existing pre-training objectives either ignore such structure or focus on learning to use it. The feasibility and benefits of learning the knowledge of analyzing semantic structure have not been investigated. To this end, this work proposes two novel pre-training objectives, namely Control Dependency Prediction (CDP) and Data Dependency Prediction (DDP), which aim to predict the statement-level control dependencies and token-level data dependencies, respectively, in a code snippet only based on its source code. During pre-training, CDP and DDP can guide the model to learn the knowledge required for analyzing fine-grained dependencies in code. After pre-training, the pre-trained model can boost the understanding of vulnerable code during fine-tuning and can directly be used to perform dependence analysis for both partial and complete functions. To demonstrate the benefits of our pre-training objectives, we pre-train a Transformer model named PDBERT with CDP and DDP, fine-tune it on three vulnerability analysis tasks, i.e., vulnerability detection, vulnerability classification, and vulnerability assessment, and also evaluate it on program dependence analysis. Experimental results show that PDBERT benefits from CDP and DDP, leading to state-of-the-art performance on the three downstream tasks. Also, PDBERT achieves F1-scores of over 99% and 94% for predicting control and data dependencies, respectively, in partial and complete functions.
@inproceedings{paperbot11413,
title = {Pre-training by Predicting Program Dependencies for Vulnerability Analysis Tasks},
author = {Zhongxin Liu and Zhijie Tang and Junwei Zhang and Xin Xia and Xiaohu Yang},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639142}
}