一句话：实证研究传统覆盖准则识别LLM越狱漏洞的有效性，并提出实时检测、测试优先级与覆盖引导攻击生成三项应用。
论文关注大语言模型在恶意攻击下行为异常、发布前测试不足的问题，以越狱攻击为切入点，系统评估传统覆盖准则能否有效识别此类缺陷。作者首先对LLM隐藏状态进行聚类分析，发现其嵌入特征能有效区分不同查询类型；随后从准则层、层、token三个维度系统评估覆盖准则的表现，发现模型处理正常查询与越狱查询时的神经元覆盖存在显著差异，与聚类结果一致。基于这些发现，论文提出覆盖准则在LLM安全测试中的三项应用：实时越狱检测机制（分类正常与越狱查询平均准确率达93.61%）、利用覆盖水平对测试用例进行优先级排序以提升测试效率并去除冗余测试、以及覆盖引导的越狱攻击样例生成方法以系统化地精化提示、暴露漏洞。该工作加深了对LLM安全测试的理解，并为构建更鲁棒的AI应用提供了基础。
Large language models (LLMs) have revolutionized artificial intelligence, but their increasing deployment across critical domains has raised concerns about their abnormal behaviors when faced with malicious attacks. Such vulnerability alerts the widespread inadequacy of pre-release testing. In this paper, we conduct a comprehensive empirical study to evaluate the effectiveness of traditional coverage criteria in identifying such inadequacies, exemplified by the significant security concern of jailbreak attacks. Our study begins with a clustering analysis of the hidden states of LLMs, revealing that the embedded characteristics effectively distinguish between different query types. We then systematically evaluate the performance of these criteria across three key dimensions: criterion level, layer level, and token level. Our research uncovers significant differences in neuron coverage when LLMs process normal versus jailbreak queries, aligning with our clustering experiments. Leveraging these findings, we propose three practical applications of coverage criteria in the context of LLM security testing. Specifically, we develop a realtime jailbreak detection mechanism that achieves high accuracy (93.61 % on average) in classifying queries as normal or jailbreak. Furthermore, we explore the use of coverage levels to prioritize test cases, improving testing efficiency by focusing on high-risk interactions and removing redundant tests. Lastly, we introduce a coverage-guided approach for generating jailbreak attack examples, enabling systematic refinement of prompts to uncover vulnerabilities. This study improves our understanding of LLM security testing, enhances their safety, and provides a foundation for developing more robust AI applications.
DOI 原文 ·
@inproceedings{paperbot8691,
title = {Understanding the Effectiveness of Coverage Criteria for Large Language Models: A Special Angle from Jailbreak Attacks},
author = {Shide Zhou and Tianlin Li and Kailong Wang and Yihao Huang and Ling Shi and Yang Liu and Haoyu Wang},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00209}
}