一句话：揭示 LLM 驱动应用的内容投毒攻击：对人类看似良性却诱导模型输出恶意响应，并系统评估其效果与现有防御局限。
本文指出，LLM 驱动应用常直接处理互联网不可信内容，却缺乏充分安全考量，从而面临“内容投毒”威胁：攻击者构造对人类看似良性、却能诱导 LLM 应用生成恶意响应的内容。作者系统分析该攻击，覆盖不同内容中的攻击模式、LLM 应用框架中可被利用的设计特性，以及攻击内容生成方法。在五个 LLM 上评估，平均攻击成功率达 89.60%；在四个流行 LLM 驱动应用中，72.00% 的内容被成功投毒。实验还表明现有防御对内容投毒无效，并讨论了面向 LLM 应用框架的潜在缓解措施。
Large Language Models (LLMs) have shown their superior capability in natural language processing, promoting extensive LLM-powered applications to be the new portals for people to access various content on the Internet. However, LLM-powered applications do not have sufficient security considerations on untrusted content, leading to potential threats. In this paper, we reveal content poisoning, where attackers can tailor attack content that appears benign to humans but causes LLM-powered applications to generate malicious responses. To highlight the impact of content poisoning and inspire the development of effective defenses, we systematically analyze the attack, focusing on the attack modes in various content, exploitable design features of LLM application frameworks, and the generation of attack content. We carry out a comprehensive evaluation on five LLMs, where content poisoning achieves an average attack success rate of 89.60%. Additionally, we assess content poisoning on four popular LLM-powered applications, achieving the attack on 72.00% of the content. Our experimental results also show that existing defenses are ineffective against content poisoning. Finally, we discuss potential mitigations for LLM application frameworks to counter content poisoning.CCS CONCEPTS• Computing methodologies → Machine learning; • Security and privacy;
@inproceedings{ZhangZGZS0024,
title = {Imperceptible Content Poisoning in LLM-Powered Applications},
author = {Quan Zhang and Chijin Zhou and Gwihwan Go and Binqi Zeng and Heyuan Shi and Zichen Xu and Yu Jiang},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695001}
}