一句话：提出 Bat 服务系统与二分注意力，通过自适应前缀选择与 KV 缓存复用，显著提升生成式推荐推理吞吐并降低计算量。
生成式推荐模型（GR）在推理时计算昂贵，呈现类似 LLM prefill 的计算受限特征；但由于用户画像高度个性化和物品实时检索，传统前缀缓存难以跨查询复用。本文提出 Bat 服务系统。其关键观察是用户 token 与物品 token 的语义具有排列不变性，据此设计二分注意力（Bipartite Attention），可在不损失精度的前提下自适应选择用户或物品作为提示前缀，从而创造 KV 缓存复用机会。系统进一步协同设计分离式 KV 缓存池，将用户前缀与物品前缀缓存分开管理；针对物品缓存的内存开销，提出热复制冷分片放置策略；并引入热度感知提示调度以在内存约束下优化前缀选择。在多个推荐数据集上，Bat 相比传统 user-as-prefix 方法将服务吞吐提升最高 1.6 倍，总计算量减少最高 58%。
Generative Recommenders (GRs) have recently emerged as promising alternatives to traditional Deep Learning Recommendation Models (DLRMs). Despite their potential, GRs remain computationally expensive in inference, exhibiting compute-bound characteristics similar to the prefill stage of Large Language Model (LLM) inference. Prefix caching can reduce redundant computation by reusing previously constructed KV caches. However, the unique properties of GRs, i.e., highly personalized user profiles and real-time item retrieval, make cache reuse across queries challenging, resulting in limited computational savings. To address these challenges, we present Bat, an efficient serving system for GRs. The key observation is that the semantics between user and item tokens are permutation-invariant. Building on this, we propose Bipartite Attention, a novel attention mechanism that enables adaptive selection of either the user or the item as the prompt prefix without compromising accuracy, thereby unlocking new opportunities for KV cache reuse. We further co-design a disaggregated KV cache pool to proactively manage user-prefix and item-prefix caches as separate components. Since introducing item caches incurs additional memory overhead, we develop a hot-replicated cold-sharded item cache placement strategy that minimizes memory usage and maintains low communication overheads. Finally, we introduce a hotness-aware prompt scheduling strategy to optimize prefix selection under memory constraints. Extensive experiments on multiple recommendation datasets demonstrate that BAT improves serving throughput by up to 1.6x over the conventional user-as-prefix approach, while reducing total computation by up to 58%.
@inproceedings{0017WZLXSZH0W26,
title = {<scp>Bat:</scp>
Efficient Generative Recommender Serving with Bipartite Attention},
author = {Jie Sun and Shaohang Wang and Zimo Zhang and Zhengyu Liu and Yunlong Xu and Peng Sun and Bo Zhao and Bingsheng He and Fei Wu and Zeke Wang},
booktitle = {ASPLOS},
year = {2026},
doi = {10.1145/3779212.3790131}
}