一句话：提出 RDMA 编程框架 SMART，隐藏 RNIC 底层细节并解决三大扩展瓶颈，大幅提升内存分离应用吞吐。
内存分离应用通过 RDMA 远程访问分离内存，但在大量并发细粒度访问下受限于网络 IOPS，现有应用难以扩展到 32 核以上。通过分析 RNIC 内部架构，发现三类扩展瓶颈：门铃寄存器隐式竞争、过多未完成工作请求导致缓存颠簸，以及失败 CAS 重试浪费 IOPS。为此提出 SMART——一个 RDMA 编程框架，以类似单边 RDMA verbs 的接口隐藏底层细节。评估中，用 44 行和 16 行代码分别重构 RACE 与 FORD，吞吐最高提升 132.4 倍与 5.2 倍；再用 SMART 和推测性查找优化重构 Sherman，仅改 48 行代码，使其访存模式从带宽受限转为 IOPS 受限，获得 2.0 倍加速。SMART 已开源。
Recent developments in RDMA networks are leading to the trend of memory disaggregation. However, the performance of each compute node is still limited by the network, especially when it needs to perform a large number of concurrent fine-grained remote accesses. According to our evaluations, existing IOPS-bound disaggregated applications do not scale well beyond 32 cores, and therefore do not take full advantage of today's many-core machines. After an in-depth analysis of the internal architecture of RNIC, we found three major scale-up bottlenecks that limit the throughput of today's disaggregated applications: (1) implicit contention of doorbell registers, (2) cache trashing caused by excessive outstanding work requests, and (3) wasted IOPS from unsuccessful CAS retries. However, the solutions to these problems involve many low-level details that are not familiar to application developers. To ease the burden on developers, we propose Smart, an RDMA programming framework that hides the above details by providing an interface similar to one-sided RDMA verbs. We take 44 and 16 lines of code to refactor the state-of-the-art disaggregated hash table (RACE) and persistent transaction processing system (FORD) with Smart, improving their throughput by up to 132.4× and 5.2×, respectively. We have also refactored Sherman (a recent disaggregated B+Tree) with Smart and an additional speculative lookup optimization (48 lines of code changed), which changes its memory access pattern from bandwidth-bound to IOPS-bound and leads to a speedup of 2.0×. Smart is publicly available at https://github.com/madsys-dev/smart.
@inproceedings{RenZCXCW24,
title = {Scaling Up Memory Disaggregated Applications with SMART},
author = {Feng Ren and Mingxing Zhang and Kang Chen and Huaxia Xia and Zuoning Chen and Yongwei Wu},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3617232.3624857}
}