一句话：提出 Bullet，通过时空编排、SLO 感知调度与自适应资源分配并发执行 prefill 与 decode，显著提升 LLM 服务 GPU 利用率与吞吐。
现代大语言模型（LLM）服务系统中，计算密集的 prefill 阶段与访存密集的 decode 阶段存在根本性不匹配，导致 GPU 利用率低下。现有混合批处理方案在吞吐与延迟之间做出低效折中：prefill 受 wave quantization 与注意力瓶颈影响计算利用率不足，而混合批处理又过度偏向延迟、浪费算力与显存带宽。为此，作者提出 Bullet——一种基于细粒度阶段协调的时空编排系统。Bullet 允许 prefill 与 decode 请求并发执行，并依据实时性能建模动态分配 GPU 资源；通过融合 SLO 感知调度与自适应资源分配，在不牺牲延迟目标的前提下最大化 GPU 利用率。在真实工作负载上的实验表明，Bullet 相比当前最优方法平均取得 1.26 倍吞吐提升（最高 1.55 倍），并持续满足延迟约束。
Modern large language model (LLM) serving systems confront inefficient GPU utilization due to the fundamental mismatch between compute-intensive prefill phase and memory-bound decode phase. While current practices attempt to address this by organizing these phases into hybrid batches, such solutions create an inefficient tradeoff that sacrifices either throughput or latency, leaving substantial GPU resources underutilized. For this, we identify two key root causes: 1) the prefill phase suffers from suboptimal compute utilization due to wave quantization and attention bottlenecks, and 2) hybrid batching disproportionately prioritizes latency over throughput, wasting both compute resources and memory bandwidth. To mitigate the issues, we present Bullet, a novel spatial-temporal orchestration system that eliminates these inefficiencies through fine-grained phase coordination. Bullet enables concurrent execution of prefill and decode requests, while dynamically provisioning GPU resources based on real-time performance modeling. By integrating SLO-aware scheduling and adaptive resource allocation, Bullet maximizes GPU utilization without compromising latency targets. Experimental evaluations on real-world workloads demonstrate that Bullet delivers 1.26× average throughput gains (up to 1.55×) over state-of-the-arts, while consistently meeting latency constraints.
@inproceedings{0001XC0L026,
title = {Bullet: Boosting GPU Utilization for LLM Serving via Dynamic Spatial-Temporal Orchestration},
author = {Zejia Lin and Hongxin Xu and Guanyi Chen and Zhiguang Chen and Yutong Lu and Xianwei Zhang},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790135}
}