paperbot · PL 论文追踪

全部论文 待读 收藏

P0 Safeguarding LLM Agents from Misalignment through Provenance Analysis

通过溯源分析保障 LLM 智能体免于失准
ASE Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering2026匹配度 8/10 · 论文将溯源分析（一种程序分析技术）形式化为 LLM Agent 工具调用前的失准检测框架，属于在 Agentic 程序中应用程序分析/形式化方法的方向，与「PL→AI」高度契合；同时包含形式化框架与工程流水线，符合用户对理论+实现的偏好。引用 6 inbox
Yining She, Yiliang Liang, Eunsuk Kang
机器学习与程序语言安全与隐私
crossref: 2026-10-09

AI 速览

一句话：提出基于溯源分析的形式化框架与 ProvenanceGuard 流水线，在工具调用执行前检测 LLM 智能体的失准行为。

随着 LLM 智能体获得越来越强的工具调用能力，其动作是否契合用户意图变得至关重要，偏离意图的失准行为可能造成难以挽回的损害。现有运行时护栏多依赖 LLM-as-a-judge 范式，缺乏系统化的对齐推理框架，判断常不一致且难以审计。本文受溯源分析启发，将失准检测形式化为判断：候选工具调用是否被智能体上下文中的可追溯证据所支持。基于该框架实现 ProvenanceGuard——一个多阶段流水线，在执行前检查三类失准，仅放行有证据支持的动作。在 AgentSafetyBench 与 WorkBench 上跨 11 个基座 LLM 评估，相较 LLM-as-a-judge 基线显著降低了失准轨迹上的错误率。

问题
LLM 智能体的工具调用可能与用户意图不一致，而现有运行时护栏依赖 LLM-as-a-judge，缺乏系统化的对齐推理框架，判断不一致且难以审计。
方法
以溯源分析为动机，将失准检测形式化为「候选工具调用是否有智能体上下文中可追溯证据支持」，并据此实现多阶段流水线 ProvenanceGuard，在执行前识别三类失准并只放行对齐动作。
评估
在 AgentSafetyBench 与 WorkBench 两个基准上，以 11 个基座 LLM 为后端进行评测，并与 LLM-as-a-judge 基线比较失准轨迹上的错误率。
原文摘要（Abstract）

As LLM agents gain increasing access to powerful tools, ensuring that their actions align with the user’s intent becomes critical. When an agent’s proposed action deviates from that intent—a phenomenon called misalignment— it may cause harm that is difficult to undo. Existing runtime guardrails rely on an LLM-as-a-judge paradigm that lacks a systematic framework for reasoning about alignment, often producing inconsistent or difficult-to-audit judgments. Motivated by provenance analysis, we propose a conceptual framework that formalizes misalignment detection as determining whether a proposed tool call is supported by traceable evidence in the agent’s context. Based on this framework, we build ProvenanceGuard, a multi-stage pipeline that analyzes the agent’s action for three types of misalignment before its execution and only allows aligned actions. We evaluated ProvenanceGuard on AgentSafetyBench and WorkBench, across 11 backbone LLMs. Compared to the LLM-as-a-judge baseline, ProvenanceGuard reduces error rate on misaligned traces from 44.3

链接与引用

DOI 原文 · arXiv · DBLP

BibTeX 
@inproceedings{abs-2607-01236,
  title = {Safeguarding LLM Agents from Misalignment through Provenance Analysis},
  author = {Yining She and Yiliang Liang and Eunsuk Kang},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3837535}
}