一句话：用 Raft 复制内核、超售资源并按需分配 GPU，大幅提升交互式深度学习训练的 GPU 利用率与响应性。
交互式 notebook 编程已成为现代机器学习与 AI 工作流的标准形态，其中交互式深度学习训练（IDLT）是主流用例。为保证响应性，Jupyter、Colab 等平台会为长时间运行的 notebook 会话长期预留 GPU，但实际 GPU 使用是间歇且零散的，造成 GPU 利用率极低、成本过高。本文提出 NotebookOS，一个面向 IDLT 特殊需求的高 GPU 效率 notebook 平台。它采用跨 GPU 服务器分布的、由 Raft 同步的复制式 notebook 内核；利用 IDLT 工作负载中较长的到达间隔，对服务器资源进行超售，并仅在单元（cell）实际执行期间分配 GPU；同时支持高负载下的副本迁移与自动集群伸缩。这些设计共同实现了低延迟的交互式训练。在生产工作负载上的评估显示，NotebookOS 在 17.5 小时的真实 IDLT 中节省了超过 1,187 GPU 小时，并显著提升了交互性。
Interactive notebook programming is universal in modern ML and AI workflows, with interactive deep learning training (IDLT) emerging as a dominant use case. To ensure responsiveness, platforms like Jupyter and Colab reserve GPUs for long-running notebook sessions, despite their intermittent and sporadic GPU usage, leading to extremely low GPU utilization and prohibitively high costs. In this paper, we introduce NotebookOS, a GPU-efficient notebook platform tailored for the unique requirements of IDLT. NotebookOS employs replicated notebook kernels with Raft-synchronized replicas distributed across GPU servers. To optimize GPU utilization, NotebookOS oversubscribes server resources, leveraging high interarrival times in IDLT workloads, and allocates GPUs only during active cell execution. It also supports replica migration and automatic cluster scaling under high load. Altogether, this design enables interactive training with minimal delay. In evaluation on production workloads, NotebookOS saved over 1,187 GPU hours in 17.5 hours of real-world IDLT, while significantly improving interactivity.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{CarverZWM026,
title = {NotebookOS: A Replicated Notebook Platform for Interactive Training with On-Demand GPUs},
author = {Benjamin Carver and Jingyuan Zhang and Haoliang Wang and Kanak Mahadik and Yue Cheng},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3760250.3762230}
}