一句话：提出 ARIA 自动化红队框架，用攻击 LLM 迭代生成隐蔽且有效的代码任务指令后门，攻击成功率最高达 0.945。
LLM 定制平台允许用户通过向系统提示词中嵌入指令来构建面向代码智能任务的专用模型，而无需修改底层参数。这种低门槛定制也引入了新的攻击面——指令后门攻击：攻击者在定制指令中植入隐藏的恶意行为。现有攻击存在两点不足：一是依赖易被平台侧或用户侧检测发现的显式触发器模式；二是需要大量人工为特定任务手工构造后门指令，难以扩展。本文提出 ARIA，一个自动化的红队测试框架，用于构造隐蔽且有效的后门指令。ARIA 借助攻击者 LLM 迭代地生成与精炼后门指令，并以目标 LLM 在隐蔽性、干净任务效用和后门有效性三个维度上的结构化反馈作为引导。作者在三个代码智能任务、四种代表性 LLM 上评测 ARIA，并与三种基线攻击对比。实验表明 ARIA 取得最高 0.945 的攻击成功率，同时在所有任务上保持最佳的干净任务效用；它还能跨编程语言泛化，对生成温度保持鲁棒，并在规避平台侧与用户侧检测方面显著优于现有攻击，假阴性率最高可达 1.000，且对现有防御方法仍然有效。
LLM customization platforms allow users to build task-specific models for code intelligence tasks by embedding instructions into system prompts, without modifying the underlying model parameters. While these platforms lower the barrier to developing customized LLMs, they also introduce a new attack surface: instruction backdoor attacks, in which adversaries implant hidden malicious behaviors into customized instructions. However, existing attacks suffer from two key limitations. First, they often rely on explicit trigger patterns readily detected by platform-side or user-side inspection. Second, they require substantial manual effort to craft task-specific backdoored instructions, limiting their scalability. In this paper, we propose ARIA, an automated red-teaming framework for crafting covert and effective backdoored instructions against customized LLMs. ARIA leverages an attacker LLM to iteratively generate and refine backdoored instructions, guided by structured feedback from the target LLM along three dimensions: stealthiness, clean-task utility, and backdoor effectiveness. We evaluate ARIA on three code intelligence tasks, using four representative LLMs, and compare it with three baseline attacks. Experimental results show that ARIA achieves the highest attack success rate of 0.945, while maintaining the best clean-task utility across all tasks. ARIA also generalizes well across programming languages and remains robust to generation temperature. Furthermore, ARIA significantly outperforms existing attacks in evading platform-side and user-side detection, achieving a false negative rate of up to 1.000, and stays effective against existing defense methods, demonstrating its strong generalizability and robustness.
@inproceedings{paperbot12237,
title = {Breaking Customized LLMs for Coding: Automated Red Teaming for Instruction Backdoor Attacks},
author = {Yuchen Chen and Wei Cheng and Yuan Xiao and Weisong Sun and Chunrong Fang and Yang Liu and Zhenyu Chen and Baowen Xu},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834424}
}