paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Think Before You Code: Dual Reasoning for the NLSafety–Utility Trade-Off in LLM Code Generation

先思考再编码：面向 LLM 代码生成中自然语言安全–效用权衡的双重推理
ASE Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering2026匹配度 2/10 · 该论文聚焦 LLM 代码生成中的安全—效用权衡，提出推理时提示/审查方法与评价指标，本质上属于以 LLM/Agent 能力为研究目标的提示工程类工作；与用户核心关注的静态分析、形式化验证、浮点程序分析、贝叶斯程序分析及其与 LLM 的交叉不匹配。引用 1 inbox
Honghao Tan, Haibo Wang, Shin Hwei Tan
机器学习与程序语言
crossref: 2026-10-09

AI 速览

一句话：提出 SUDS 指标与 Dual Reasoning 推理时方法，在代码生成前做安全审计与审查，统一功能正确性与自然语言安全警告。

现有 Code LLM 仅以功能正确性为评估标准，忽视生成代码是否传播提示中注入的有害内容；已有方法多聚焦有害内容检测，却忽略功能正确性。本文基于双重信道约束理论（代码同时具有面向机器执行的算法信道与面向人类交流的自然语言信道，因而存在独特的安全–效用权衡），提出 NLSafety-Utility Duality Score（SUDS），在 12 种排序响应场景下把代码效用、安全遵循与警告意识统一为单一分数；并提出 Dual Reasoning（DR），一种要求在生成代码前显式进行安全审计与任务相关代码审查的结构化推理时技术。在 5 个 LLM 与 2 个注入有害关键词的基准（820 与 2135 个样本）上，DR 在所有模型上均取得最高 SUDS，较基线平均提升 1.32×–3.42×；而思维链提示的安全增益微乎其微，安全感知提示仅有部分改善。进一步分析表明 DR 的效果随模型容量扩展，one-shot 示例主要用于稳定小模型的输出格式，结构化推理无法弥补安全词表有限的模型。

问题
现有 Code LLM 评估只看功能正确性，忽视生成代码是否会复现提示中注入的有害标识符；已有安全方法侧重有害内容检测，却牺牲了功能正确性。
方法
基于双信道约束理论提出 SUDS 指标，在 12 种排序响应场景下把代码效用、安全遵循与警告意识统一为单一分数；并提出 Dual Reasoning，一种在生成代码前强制执行显式安全审计与任务相关代码审查的结构化推理时技术。
评估
在 5 个 LLM 与 2 个注入有害关键词的基准（820 与 2135 个样本）上评估，DR 在所有模型上取得最高 SUDS，平均较基线提升 1.32×–3.42×，并进一步分析模型容量、one-shot 示例与安全词表对效果的影响。
原文摘要（Abstract）

Large language models (LLMs) for code generation are typically evaluated on functional correctness alone, overlooking whether generated code propagates harmful content embedded in the prompt. Prior work has shown that most Code LLMs reproduce offensive identifiers from injected renaming instructions without warning, yet existing approaches focus on detecting harmful content, neglecting functional correctness. Grounded in the Theory of Dual Channel Constraints (which states that code is a dual-channel medium combining an algorithmic (AL) channel for machine execution and a natural language (NL) channel for human communication, creating a unique safety-utility trade-off where a model must balance functional execution with responsible communication), we propose NLSafety-Utility Duality Score (SUDS), a metric that unifies code utility, safety adherence, and warning awareness into a single score across 12 ranked response scenarios, and Dual Reasoning (DR), a structured inference-time technique that requires an explicit safety audit and task-grounded code review before code generation. Evaluated on five LLMs across two benchmarks augmented with harmful keyword injections (820 and 2,135 samples), DR consistently achieves the highest SUDS across all models, improving mean SUDS by 1.32× to 3.42× over the baseline, while chain-of-thought prompting yields negligible safety gains and a safety-aware prompt provides only partial improvement. Further analysis reveals that DR's effectiveness scales with model capacity, that the one-shot exemplar primarily stabilizes output format for smaller models, and that structured reasoning cannot compensate for models with limited safety vocabularies.

链接与引用

DOI 原文 · arXiv ·

BibTeX 
@inproceedings{paperbot12239,
  title = {Think Before You Code: Dual Reasoning for the NLSafety–Utility Trade-Off in LLM Code Generation},
  author = {Honghao Tan and Haibo Wang and Shin Hwei Tan},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3834385}
}