一句话：提出内省弹性与 Dilu 系统，以细粒度二维协同伸缩和互补调度降低无服务器 DL 的 GPU 碎片并保障 QoS。
无服务器 DL 因负载动态和粗粒度静态 GPU 分配易产生 15%–94% 的 GPU 碎片，削弱弹性并威胁 QoS。本文提出内省弹性（IE），一种细粒度、自适应二维协同伸缩机制，并构建跨层 GPU 无服务器 DL 系统 Dilu：它提供多因素 profiling 与高效剪枝搜索，在调度中遵循资源互补原则，并实时自适应二维协同伸缩 GPU 供给。评估表明，相比 SOTA，Dilu 可将 GPU 碎片降低 10%–46%，推理吞吐最高提升 1.8 倍、训练吞吐提升 1.1 倍，QoS 违规率降低 11%–71%。
Serverless computing, with its ease of management, auto-scaling, and cost-effectiveness, is widely adopted by deep learning (DL) applications. DL workloads, especially with large language models, require substantial GPU resources to ensure QoS. However, it is prone to produce GPU fragments (e.g., 15%-94%) in serverless DL systems due to the dynamicity of workloads and coarse-grained static GPU allocation mechanisms, gradually eroding the profits offered by serverless elasticity. Different from classical serverless systems that only scale horizontally, we present introspective elasticity (IE), a fine-grained and adaptive two-dimensional co-scaling mechanism to support GPU resourcing-on-demand for serverless DL tasks. Based on this insight, we build Dilu, a cross-layer and GPU-based serverless DL system with IE support. First, Dilu provides multi-factor profiling for DL tasks with efficient pruning search methods. Second, Dilu adheres to the resourcing-complementary principles in scheduling to improve GPU utilization with QoS guarantees. Third, Dilu adopts an adaptive 2D co-scaling method to enhance the elasticity of GPU provisioning in real time. Evaluations show that it can dynamically adjust the resourcing of various DL functions with low GPU fragmentation (10%-46% GPU defragmentation), high throughput (up to 1.8× inference and 1.1× training throughput increment) and QoS guarantees (11%-71% violation rate reduction), compared to the SOTA baselines.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{Lv0LHTZZ25,
title = {Dilu: Enabling GPU Resourcing-on-Demand for Serverless DL Serving via Introspective Elasticity},
author = {Cunchi Lv and Xiao Shi and Zhengyu Lei and Jinyue Huang and Wenting Tan and Xiaohui Zheng and Xiaofang Zhao},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3669940.3707251}
}