一句话:提出面向一致性互连的主机-NIC接口CC-NIC,显著降低延迟并提升吞吐。
新兴互连让NIC等外设可通过处理器缓存层次访问并参与缓存一致性协议,这与当前PCIe NIC的独立I/O数据路径和读写事务原语有根本差异。作者发现现有I/O路径令NIC以延迟膨胀换取CPU效率,而一致性抽象并不适配现有主机-NIC访问模式:沿用已有信令机制和数据结构布局会带来多余通信与缓存驻留,限制性能。为此提出CC-NIC,一种面向一致性互连的主机-NIC接口设计,并在Intel Ice Lake与Sapphire Rapids UPI互连上建模评估。结果表明,CC-NIC最高包速率达1.5Gpps、包吞吐达980Gbps;相比当今PCIe NIC,最小延迟降低77%,80%负载下降低88%,并带来应用级核心节省;其收益在多种互连性能特征下保持。
Emerging interconnects make peripherals, such as the network interface controller (NIC), accessible through the processor's cache hierarchy, allowing these devices to participate in the CPU cache coherence protocol. This is a fundamental change from the separate I/O data paths and read-write transaction primitives of today's PCIe NICs. Our experiments show that the I/O data path characteristics cause NICs to prioritize CPU efficiency at the expense of inflated latency, an issue that can be mitigated by the emerging low-latency coherent interconnects. But, the coherence abstraction is not suited to current host-NIC access patterns. Applying existing signaling mechanisms and data structure layouts in a cache-coherent setting results in extraneous communication and cache retention, limiting performance. Redesigning the interface is necessary to minimize overheads and benefit from the new interactions coherence enables. This work contributes CC-NIC, a host-NIC interface design for coherent interconnects. We model CC-NIC using Intel's Ice Lake and Sapphire Rapids UPI interconnects, demonstrating the potential of optimizing for coherence. Our results show a maximum packet rate of 1.5Gpps and 980Gbps packet throughput. CC-NIC has 77% lower minimum latency, and 88% lower at 80% load, than today's PCIe NICs. We also demonstrate application-level core savings. Finally, we show that CC-NIC's benefits hold across a range of interconnect performance characteristics.
@inproceedings{SchuhKCLRKS24,
title = {CC-NIC: a Cache-Coherent Interface to the NIC},
author = {Henry N. Schuh and Arvind Krishnamurthy and David Culler and Henry M. Levy and Luigi Rizzo and Samira Khan and Brent E. Stephens},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3617232.3624868}
}