一句话：提出内容感知的资源管理方法 AIMS，通过预测请求开销实现动态主动自动扩缩容，降低 SLO 违规并提升资源利用率。
AI 助手系统（如 Gemini、Dola）任务处理能力强，但资源开销巨大，资源管理不当会导致延迟尖峰和 SLO 违规。传统资源管理主要监控用户请求速率，忽略了请求内容对系统工作负载的关键影响，因此难以适用于此类环境。本文提出 AIMS，一种面向 AI 助手系统的内容感知资源管理方法。它分析用户请求内容以预测工作负载，并通过预测到达请求的资源开销，支持动态、主动的自动扩缩容。基于大规模生产 AI 助手系统的分析表明，AIMS 能显著减少 SLO 违规，同时提升整体资源利用率。
AI Assistant Systems, such as Gemini and Dola, have become integral to many applications due to their sophisticated task-handling capabilities. However, these systems incur substantial resource overheads, and suboptimal resource management can cause latency spikes and violations of Service Level Objectives (SLOs). Traditional resource management methods, which primarily monitor user request rate, are inadequate for these environments because they overlook the critical impact of user request content on system workloads. This paper introduces AIMS, a content-aware resource management approach designed specifically for AI Assistant Systems. Unlike conventional approaches, AIMS analyzes user request content to forecast the workload. By predicting the resource overheads of incoming requests, AIMS enables dynamic and proactive autoscaling. Our analysis of a large-scale, production AI Assistant System demonstrates that AIMS can significantly reduce SLO violations while improving overall resource utilization.
@inproceedings{DuanJHXZZZL26,
title = {AIMS: A Content-Aware Resource Management Approach for AI Assistant Systems},
author = {Chiming Duan and Tong Jia and Minghua He and Pei Xiao and Lingzhe Zhang and Zhewei Zhong and Xin Zhang and Ying Li},
booktitle = {FSE},
year = {2026},
doi = {10.1145/3803437.3805562}
}