一句话：通过剖析RAG流水线，提出基于CXL近内存加速的IKS，加速精确最近邻检索并降低端到端推理时间。
针对大语言模型幻觉与准确性问题，检索增强生成（RAG）通过外部知识源增强生成，但其检索与生成阶段相互影响，且精确检索方案代价高。本文对多个RAG执行流水线进行剖析，发现精确检索虽单次开销大，却能以更小且更准确的文档列表降低生成阶段推理时间。为此，作者设计 Intelligent Knowledge Store（IKS），一种 type-2 CXL 设备，采用面向近内存加速器的 scale-out 架构，并在宿主 CPU 与近内存加速器之间提供缓存一致性接口。在 512GB 向量数据库上，IKS 的精确最近邻搜索比 Intel Sapphire Rapids CPU 快 13.4–27.9 倍，使代表性 RAG 应用的端到端推理时间降低 1.7–26.3 倍；其内部 DRAM 还可作为内存扩展器被解聚复用。
An evolving solution to address hallucination and enhance accuracy in large language models (LLMs) is Retrieval-Augmented Generation (RAG), which involves augmenting LLMs with information retrieved from an external knowledge source, such as the web. This paper profiles several RAG execution pipelines and demystifies the complex interplay between their retrieval and generation phases. We demonstrate that while exact retrieval schemes are expensive, they can reduce inference time compared to approximate retrieval variants because an exact retrieval model can send a smaller but more accurate list of documents to the generative model while maintaining the same end-to-end accuracy. This observation motivates the acceleration of the exact nearest neighbor search for RAG. In this work, we design Intelligent Knowledge Store (IKS), a type-2 CXL device that implements a scale-out near-memory acceleration architecture with a novel cache-coherent interface between the host CPU and near-memory accelerators. IKS offers 13.4--27.9× faster exact nearest neighbor search over a 512GB vector database compared with executing the search on Intel Sapphire Rapids CPUs. This higher search performance translates to 1.7--26.3× lower end-to-end inference time for representative RAG applications. IKS is inherently a memory expander; its internal DRAM can be disaggregated and used for other applications running on the server to prevent DRAM -- which is the most expensive component in today's servers -- from being stranded.
@inproceedings{abs-2412-15246,
title = {Accelerating Retrieval-Augmented Generation},
author = {Derrick Quinn and Mohammad Nouri and Neel Patel and John Salihu and Alireza Salemi and Sukhan Lee and Hamed Zamani and Mohammad Alian},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3669940.3707264}
}