一句话：提出 IOCost，面向数据中心容器提供可扩展、工作守恒、低开销的块 IO 控制与比例资源隔离。
本文针对 Meta 大规模数据中心中容器化环境下现有块存储 IO 控制不足的问题，提出 IOCost。挑战包括存储设备异构、工作负载差异、SSD 高速带来的低开销需求，以及与内存管理子系统的交互和优先级反转导致的隔离失效。IOCost 通过离线剖析构建设备模型，估计每个 IO 请求的设备占用；运行时分为快速的逐 IO 发放路径与较慢的周期性规划路径以降低开销，并采用工作守恒的预算捐赠算法让容器动态共享未用预算。IOCost 已全面部署于 Meta 数百万台机器，上游合入 Linux 内核，开源设备剖析工具，并在生产中运行两年。
Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale.
@inproceedings{HeoSNLDNBMT022,
title = {IOCost: block IO control for containers in datacenters},
author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos},
booktitle = {ASPLOS},
year = {2022},
doi = {10.1145/3503222.3507727}
}