一句话：提出执行感知的源码预训练方法 TRACED，将分支覆盖、运行时变量值等动态执行信息融入预训练，提升代码理解任务效果。
此前面向源代码的预训练语言模型主要学习静态代码文本，常辅以抽象语法树、依赖图等静态结构。但程序语义只有在真实执行中才充分暴露：缺乏执行理解的静态预训练模型难以刻画分支覆盖、运行时变量值等动态性质，因而在语义克隆检索、漏洞检测等代码理解任务上效果受限。为此作者提出 TRACED：一种执行感知的源代码预训练方法，将程序执行信息（如分支覆盖与运行时变量值）引入预训练，使模型兼顾静态与动态代码属性。评估表明，执行感知预训练在语义克隆检索与漏洞检测等任务上优于仅使用静态信息的基线。
Most existing pre-trained language models for source code focus on learning the static code text, typically augmented with static code structures (abstract syntax tree, dependency graphs, etc.). However, program semantics will not be fully exposed before the real execution. Without an understanding of the program execution, statically pre-trained models fail to comprehensively capture the dynamic code properties, such as the branch coverage and the runtime variable values, and they are consequently less effective at code understanding tasks, such as retrieving semantic clones and detecting software vulnerabilities.
DOI 原文 ·
@inproceedings{paperbot11404,
title = {TRACED: Execution-aware Pre-training for Source Code},
author = {Yangruibo Ding and Benjamin Steenhoek and Kexin Pei and Gail Kaiser and Wei Le and Baishakhi Ray},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3608140}
}