一句话：研究小代码模型对抗样本向大语言模型的迁移性，提出基于LLM的扰动生成攻击及无需重训练的提示防御。
现代大语言模型（如ChatGPT）在编码任务上表现优异，但此前的小型代码模型（如code2seq、seq2seq）已被证明易受对抗样本攻击——即微小的语法扰动即可欺骗模型。本文首先研究通过白盒攻击在小代码模型上生成的对抗样本向LLM的迁移性，并提出一种利用LLM生成扰动的新攻击方法。此外，提出基于提示的高效防御技术，无需重新训练模型，通过修改提示以包含额外信息（如对抗扰动代码示例和显式逆向扰动指令）来抵御攻击。在GPT-3.5和GPT-4上的初步实验表明，所提攻击与防御均有效。
Modern large language models (LLMs), such as ChatGPT, have demonstrated impressive capabilities for coding tasks, including writing and reasoning about code. They improve upon previous neural network models of code, such as code2seq or seq2seq, that already demonstrated competitive results when performing tasks such as code summarization and identifying code vulnerabilities. However, these previous code models were shown vulnerable to adversarial examples, i.e., small syntactic perturbations designed to "fool" the models. In this paper, we first aim to study the transferability of adversarial examples, generated through white-box attacks on smaller code models, to LLMs. We also propose a new attack using an LLM to generate the perturbations. Further, we propose novel cost-effective techniques to defend LLMs against such adversaries via prompting, without incurring the cost of retraining. These prompt-based defenses involve modifying the prompt to include additional information, such as examples of adversarially perturbed code and explicit instructions for reversing adversarial perturbations. Our preliminary experiments show the effectiveness of the attacks and the proposed defenses on popular LLMs such as GPT-3.5 and GPT-4.
@inproceedings{paperbot8380,
title = {Attacks and Defenses for Large Language Models on Coding Tasks},
author = {Chi Zhang and Zifan Wang and Ruoshi Zhao and Ravi Mangal and Matt Fredrikson and Limin Jia and Corina Pasareanu},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695297}
}