一句话:提出 AgentOS,用类型化工具中介 VTRM 与宿主侧世界效应评判器,在保持良性任务效用的同时阻断 LLM 智能体的有害世界效应。
LLM 智能体日益作为可执行动作的系统,对文件、数据库、网络和外部服务进行操作,其安全失败不再只是不安全文本,而可能造成未授权的世界效应,如数据删除、受保护资源篡改或秘密外泄。本文提出 AgentOS,一个基于 Buildroot 的 LLM 智能体安全执行环境,其核心组件 Verified Toolchain Reference Monitor(VTRM)在执行前中介模型生成的工具请求,将不可信模型输出转换为类型化请求,并对文件系统、数据库和网络操作实施能力感知与标签感知的决策。AgentOS 还包含宿主侧 World Effect Judge(WEJ),依据文件系统快照、数据库差异、审计日志与网络观测而非模型自报来评估任务完成与安全性。作者在两个 LLM 上使用良性任务与对抗任务套件进行评估:在良性任务效用上 AgentOS 与仅提示词及朴素基线相当;在对抗任务上 AgentOS 阻断全部已评估的有害案例,而两个基线遗留大量未阻断的不安全效应。作者还用 TLA+ 建模 VTRM 核心决策路径并检查代表性的机密性与 fail-closed 性质。
LLM-based agents are increasingly used as action-taking systems that operate over files, databases, networks, and external services. In this setting, safety failures can become unauthorized world effects, such as data deletion, protected-resource modification, or secret exfiltration, rather than merely unsafe text outputs. We present AgentOS, a Buildroot-based execution environment for safe tool-using LLM agents. Its core component, the Verified Toolchain Reference Monitor (VTRM), mediates model-generated tool requests before execution by converting untrusted model outputs into typed requests and enforcing capability-aware and label-aware decisions over filesystem, database, and network operations. AgentOS further includes a host-side World Effect Judge (WEJ) for evaluation. WEJ measures task completion and safety using filesystem snapshots, database diffs, audit logs, and network observations rather than model self-report. We evaluate AgentOS on benign and adversarial task suites using two LLMs. Across both models, AgentOS matches prompt-only and naive baselines on benign-task utility. On adversarial tasks, AgentOS blocks all evaluated harmful cases, while both baselines leave substantially more unsafe effects unblocked. We also model the core VTRM decision path in TLA+ and check representative confidentiality and fail-closed properties. These results suggest that typed tool mediation and host-side world-effect evaluation can improve LLM-agent execution safety while preserving benign-task utility. The source code and documentation are publicly available under the MIT License at https://github.com/zhoujune/AgentOS, with an archived artifact at https://doi.org/10.5281/zenodo.20046861; the demonstration screencast is available at https://youtu.be/bqJsTRJbIkU.