一句话:提出基于机制可解释性的方法,在推理时拦截并修改 LLM 中诱发漏洞的知识表示,提升生成代码安全性。
大型语言模型(LLM)在自动代码生成上进展显著,但常生成含安全漏洞的代码。现有微调与提示工程等改进手段效果有限,且难以揭示漏洞产生的内部机制。本文提出一种基于机制可解释性的方法:先分析 LLM 内部知识,识别并解耦与生成漏洞代码相关的知识表示;再在推理阶段实时修复模型执行——当模型试图访问这些诱发漏洞的表示时,拦截并修改该访问。作者称该方法能提升生成代码的安全性,但摘要未报告具体数据集、指标或对比结果。
While automatic code generation using Large Language Models (LLMs) has advanced significantly, these models frequently produce code containing security vulnerabilities. Existing approaches to improve the security of automatically generated code, such as fine-tuning or prompt engineering, have shown limited success and provide minimal insight into the underlying mechanisms causing these vulnerabilities. We propose an approach grounded in mechanistic interpretability to analyze and mitigate vulnerable code generation in LLMs. We begin by examining the knowledge stored inside LLMs, identifying and disentangling knowledge representations that contribute to generating vulnerable code. Next, we leverage these insights to repair model execution in real time: when the model attempts to access vulnerability-inducing representations during inference, our method intercepts and modifies this access, improving the security of the generated code.