一句话：提出无损压缩框架 ZipServ，用张量核感知定长编码与融合解压-GEMM 内核，在 GPU 上兼顾 LLM 存储节省与推理加速。
无损模型压缩有望缓解位精确（bit-exact）大语言模型服务中的内存与带宽瓶颈，但现有方法因与 GPU 架构的根本性设计错配而导致推理明显变慢：在内核层面，传统熵编码产生的变长比特流破坏了 SIMT 并行；在系统层面，解耦的流水线带来冗余内存流量。本文提出 ZipServ——一个与高效 LLM 推理协同设计的无损压缩框架。它引入张量核感知的三位图编码（TCA-TBE），这是一种新的定长格式，可实现常数时间的并行解码；并配合融合的解压-GEMM 内核 ZipGEMM，将权重即时解压并直接送入 Tensor Core 寄存器。这种“加载压缩数据、计算时解压”的设计消除了中间缓冲区并最大化计算强度。实验表明，ZipServ 将模型体积最多减少 30%，内核级相较 NVIDIA cuBLAS 最高加速 2.21×，端到端推理相较 vLLM 平均加速 1.22×，是首个在 GPU 上同时提供存储节省与显著加速的无损压缩系统。
Lossless model compression holds tremendous promise for alleviating the memory and bandwidth bottlenecks in bit-exact Large Language Model (LLM) serving. However, existing approaches often result in substantial inference slowdowns due to fundamental design mismatches with GPU architectures: at the kernel level, variable-length bitstreams produced by traditional entropy codecs break SIMT parallelism; at the system level, decoupled pipelines lead to redundant memory traffic. We present ZipServ, a lossless compression framework co-designed for efficient LLM inference. ZipServ introduces Tensor-Core-Aware Triple Bitmap Encoding (TCA-TBE), a novel fixed-length format that enables constant-time, parallel decoding, together with a fused decompression-GEMM (ZipGEMM) kernel that decompresses weights on-the-fly directly into Tensor Core registers. This '' load-compressed, compute-decompressed '' design eliminates intermediate buffers and maximizes compute intensity. Experiments show that ZipServ reduces the model size by up to 30%, achieves up to 2.21× kernel-level speedup over NVIDIA's cuBLAS, and expedites end-to-end inference by an average of 1.22× over vLLM. ZipServ is the first lossless compression system that provides both storage savings and substantial acceleration for LLM inference on GPUs.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{FanYPLLW0026,
title = {Z
<scp>ip</scp>
S
<scp>erv</scp>
: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression},
author = {Ruibo Fan and Xiangrui Yu and Xinglin Pan and Zeyu Li and Weile Luo and Qiang Wang and Wei Wang and Xiaowen Chu},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790250}
}