一句话：提出基于历史窗口仲裁授权的加权仲裁，消除 NUMA 服务器处理器互连中远端节点带宽偏高的全局不公平性。
NUMA 服务器通过处理器互连（如 Intel QPI、AMD HyperTransport）连接多路节点，其仲裁会影响整体性能。本文研究处理器互连仲裁导致的性能不公平，发现与多跳网络中近端节点获益的常见认知相反，商用 NUMA 服务器中远端节点反而获得更高带宽。作者将原因定位为路由器微架构中的 external concentration 与缺乏全局感知仲裁，并展示其在 Xen 虚拟化平台上造成性能波动。为排除共享 LLC 和主存影响，使用合成负载分析。进而提出基于历史的仲裁：记录先前历史窗口内的仲裁授权，并据此加权仲裁以实现全局公平。模拟表明，该方法能以低开销提供全局公平并减小互连性能不公平。
NUMA (non-uniform memory access) servers are commonly used in high-performance computing and datacenters. Within each server, a processor-interconnect (e.g., Intel QPI, AMD HyperTransport) is used to communicate between the different sockets or nodes. In this work, we explore the impact of the processor-interconnect on overall performance -- in particular, the performance un- fairness caused by processor-interconnect arbitration. It is well known that locally-fair arbitration does not guarantee globally-fair bandwidth sharing as closer nodes receive more bandwidth in a multi-hop network. However, this work demonstrates that the opposite can occur in a commodity NUMA server where remote nodes receive higher bandwidth (and perform better). We analyze this problem and iden- tify that this occurs because of external concentration used in router micro-architectures for processor-interconnects without globally-aware arbitration. While accessing remote memory can occur in any NUMA system, performance un- fairness (or performance variation) is more critical in cloud computing and virtual machines with shared resources. We demonstrate how this unfairness creates significant performance variation when a workload is executed on the Xen virtualization platform. We then provide analysis using synthetic workloads to better understand the source of unfair- ness and eliminate the impact of other shared resources, including the shared last-level cache and main memory. To provide fairness, we propose a novel, history-based arbitration that tracks the history of arbitration grants made in the previous history window. A weighted arbitration is done based on the history to provide global fairness. Through simulations, we show our proposed history-based arbitration can provide global fairness and minimize the processor- interconnect performance unfairness at low cost.
@inproceedings{SongKJCALK17,
title = {History-Based Arbitration for Fairness in Processor-Interconnect of NUMA Servers},
author = {Wonjun Song and Gwangsun Kim and Hyungjoon Jung and Jongwook Chung and Jung Ho Ahn and Jae W. Lee and John Kim},
booktitle = {ASPLOS},
year = {2017},
doi = {10.1145/3037697.3037753}
}