一句话：提出 AdvPro，用梯度归因定位关键代码并结合 14 种保语义变异策略，自动生成对抗代码提示攻击代码补全模型。
代码补全模型在提升开发效率的同时也带来安全风险：它们可能无意引入漏洞，或被对抗性代码提示诱导生成不安全代码，而已有研究多聚焦鲁棒性，安全性分析尚不充分。本文提出 AdvPro，一种可自动生成对抗代码提示的方法。该方法在五个粒度层级上集成 14 种代码变异策略，并保证变异不改变代码语义，使其对模型而言本应不敏感；同时利用梯度归因定位代码中的重要位置作为变异点，以加速对抗提示生成。作者在 7 个家族的 13 个先进模型上开展实验，结果显示所生成对抗提示相较基线 ALERT 使攻击成功率提升 69.6%；归因定位对比表明不同模型对输入代码重要 token 的识别结果几乎一致，从而降低了用开源替代模型指导针对闭源模型攻击的限制；消融实验还发现代码补全模型重点关注变量名，但其他结构同样关键。
Large language models have made significant progress in code completion, which may further remodel future software development. However, these code completion models are found to be highly risky as they may introduce vulnerabilities unintentionally or be induced by a special input, i.e., adversarial code prompt. Prior studies mainly focus on the robustness of these models, but their security has not been fully analyzed.In this paper, we propose a novel approach AdvPro that can automatically generate adversarial code prompts for these code completion models. AdvPro incorporates 14 code mutation strategies at the granularity of five levels. The mutation strategies are ensured to make no modifications to code semantics, which should be insensitive to the models. Moreover, we leverage gradient attribution to localize the important code as mutation points and speed up adversarial prompt generation. Extensive experiments are conducted on 13 state-of-the-art models belonging to 7 families. The results show that our approach can effectively generate adversarial prompts, with an increased rate of 69.6% beyond the baseline ALERT. By comparing the results of attribution-guided localization, we find that the recognition results of important tokens in input codes are almost identical among different models. This finding reduces the limitation of using open-source alternative models to guide adversarial attacks against closed-source models. The results of the ablation study on the components of AdvPro show that CCMs focus on variable names, but other structures are equally crucial.CCS CONCEPTS• Software and its engineering → Software testing and debugging; • Security and privacy → Software security engineering.
@inproceedings{LiMLXS0L024,
title = {Attribution-guided Adversarial Code Prompt Generation for Code Completion Models},
author = {Xueyang Li and Guozhu Meng and Shangqing Liu and Lu Xiang and Kun Sun and Kai Chen and Xiapu Luo and Yang Liu},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695517}
}