一句话：揭示神经代码生成系统的用户反馈机制可被反馈数据注入攻击，并实现提示注入与后门攻击。
神经代码生成系统通常以服务形式向用户提供预训练模型，并通过反馈机制收集用户对生成代码的反应。本文首次系统研究该反馈机制的安全影响，发现其易受反馈数据注入（FDI）攻击。作者讨论了 FDI 攻击方法，并提出一种攻击前画像策略，用于在黑盒设置下推断目标系统的攻击约束。基于该攻击面，论文给出两个概念验证：对实际神经代码生成系统实施提示注入攻击和后门攻击。攻击者可隐蔽地操纵系统生成含漏洞、攻击载荷以及恶意或垃圾信息的代码。研究揭示了反馈机制在神经代码生成系统中的安全风险，为提升其安全性提供了方向。
Neural code generation systems have recently attracted increasing attention to improve developer productivity and speed up software development. Typically, these systems maintain a pre-trained neural model and make it available to general users as a service (e.g., through remote APIs) and incorporate a feedback mechanism to extensively collect and utilize the users' reaction to the generated code, i.e., user feedback. However, the security implications of such feedback have not yet been explored. With a systematic study of current feedback mechanisms, we find that feedback makes these systems vulnerable to feedback data injection (FDI) attacks. We discuss the methodology of FDI attacks and present a pre-attack profiling strategy to infer the attack constraints of a targeted system in the black-box setting. We demonstrate two proof-of-concept examples utilizing the FDI attack surface to implement prompt injection attacks and backdoor attacks on practical neural code generation systems. The attacker may stealthily manipulate a neural code generation system to generate code with vulnerabilities, attack payload, and malicious and spam messages. Our findings reveal the security implications of feedback mechanisms in neural code generation systems, paving the way for increasing their security.
@inproceedings{paperbot8462,
title = {FDI: Attack Neural Code Generation Systems through User Feedback Channel},
author = {Zhensu Sun and Xiaoning Du and Xiapu Luo and Fu Song and David Lo and Li Li},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3680300}
}