一句话：提出 MuxWise，以 GPU 内预填充-解码多路复用，在满足 SLO 下显著提升 LLM 服务吞吐。
LLM 服务需同时满足预填充与解码阶段的严格 SLO。现有方案或将两阶段解耦，造成资源闲置与计算冗余；或将预填充分块并与解码迭代融合，导致 SLO 合规与高利用率难以兼顾。本文提出 MuxWise，采用 GPU 内预填充-解码多路复用新范式，并设计无气泡多路复用引擎、竞争容忍估计器与 SLO 感知调度器，以动态适配计算分配、解耦计算与内存管理并独立执行两阶段。评估表明，在满足 SLO 保证下，MuxWise 的峰值吞吐相较最优基线平均提升 2.20 倍，最高 3.06 倍。
Large Language Model (LLM) serving must meet stringent Service Level Objectives (SLOs) for both the prefill and decode phases. Some existing solutions disaggregate the two phases, causing potential resource idleness or compute redundancy. Others split the prefill phase into chunks and fuse it with decode iteration, creating a dilemma between SLO compliance and high utilization. To address these issues, an efficient serving system should dynamically adapt compute allocation, decouple compute from memory management, and execute prefill and decode independently. We present MuxWise, an LLM serving framework that adopts a new paradigm, intra-GPU prefill-decode multiplexing, to meet these requirements. To fully exploit the paradigm, MuxWise integrates a bubble-less multiplex engine, a contention-tolerant estimator, and an SLO-aware dispatcher. Evaluation shows that MuxWise improves peak throughput under SLO guarantees by an average of 2.20x (up to 3.06x) over state-of-the-art baselines.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{ChenCZXFC0SHC26,
title = {Towards High-Goodput LLM Serving with Prefill-decode Multiplexing},
author = {Yukang Chen and Weihao Cui and Han Zhao and Ziyi Xu and Xiaoze Fan and Xusheng Chen and Yangjie Zhou and Shixuan Sun and Bingsheng He and Quan Chen},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790236}
}