paperbot · PL 论文追踪

全部论文 待读 收藏

P2 PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption

PipeLLM：基于推测式流水线加密的快速机密大语言模型服务
ASPLOS Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 12025匹配度 2/10 · 论文聚焦于 GPU 机密计算下 LLM 服务的加密开销优化，属于系统与安全方向，未涉及静态分析、形式化验证、浮点或贝叶斯程序分析，与用户的核心研究方向匹配度低。引用 33 inbox
Yifan Tan, Cheng Tan, Zeyu Mi, Haibo Chen
安全与隐私机器学习与程序语言
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话：把加密与 GPU 计算流水线化并用推测式预测待加密数据，几乎无开销地实现机密 LLM 服务。

GPU 机密计算（如 NVIDIA H100）通过强隔离与数据加密来缓解外包大语言模型（LLM）服务的安全风险，但加密带来显著的性能开销：服务 OPT-30B 与 OPT-66B 时吞吐分别下降最高 52.8% 与 88.2%。为此作者提出 PipeLLM——一个对用户透明的运行时系统。其核心思想借鉴 CPU 指令流水线，将加密与 GPU 计算重叠执行，从而把加密带来的延迟增长隐藏起来。主要技术难点在于：与 CPU 不同，加密模块在 GPU 真正发起请求之前并不知道哪些数据需要加密。为此论文提出推测式流水线加密，通过分析 LLM 的服务访问模式来预测需要加密的数据，并针对预测错误设计了一种高效、低成本的流水线放弃机制。实验表明，相比不含机密计算的基线系统（vLLM、PEFT、FlexGen），PipeLLM 在 13B 至 175B 的各规模模型上仅引入有限开销（吞吐损失小于 19.6%），源码已开源。

问题
GPU 机密计算为外包 LLM 服务提供隔离与加密保护，但加密开销使吞吐最高下降 52.8%（OPT-30B）与 88.2%（OPT-66B），需要在保持机密性的同时消除这一性能损失。
方法
PipeLLM 是一个用户透明的运行时系统，将加密与 GPU 计算流水线化以隐藏加密延迟；由于加密模块事先不知道哪些数据会被 GPU 请求，它通过分析 LLM 服务模式进行推测式流水线加密，并为错误预测提供低成本的流水线放弃机制。
评估
在 13B 到 175B 的多种 LLM 规模上，与不含机密计算的基线（vLLM、PEFT、FlexGen）对比吞吐开销，结果显示开销低于 19.6%。
原文摘要（Abstract）

Confidential computing on GPUs, like NVIDIA H100, mitigates the security risks of outsourced Large Language Models (LLMs) by implementing strong isolation and data encryption. Nonetheless, this encryption incurs a significant performance overhead, reaching up to 52.8% and 88.2% throughput drop when serving OPT-30B and OPT-66B, respectively. To address this challenge, we introduce PipeLLM, a user-transparent runtime system. PipeLLM removes the overhead by overlapping the encryption and GPU computation through pipelining-an idea inspired by the CPU instruction pipelining-thereby effectively concealing the latency increase caused by encryption. The primary technical challenge is that, unlike CPUs, the encryption module lacks prior knowledge of the specific data needing encryption until it is requested by the GPUs. To this end, we propose speculative pipelined encryption to predict the data requiring encryption by analyzing the serving patterns of LLMs. Further, we have developed an efficient, low-cost pipeline relinquishing approach for instances of incorrect predictions. Our experiments show that compared with vanilla systems without confidential computing (e.g., vLLM, PEFT, and FlexGen), PipeLLM incurs modest overhead ( < 19.6% in throughput) across various LLM sizes, from 13B to 175B. PipeLLM's source code is available at https://github.com/SJTU-IPADS/PipeLLM.

链接与引用

DOI 原文 · arXiv · PDF（开放获取） · DBLP

BibTeX 
@inproceedings{abs-2411-03357,
  title = {PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption},
  author = {Yifan Tan and Cheng Tan and Zeyu Mi and Haibo Chen},
  booktitle = {ASPLOS},
  year = {2025},
  doi = {10.1145/3669940.3707224}
}