一句话：为 Ascend NPU 算子建立性能/功耗模型，并用细粒度 DVFS 搜索在低性能损失下降低 AI 加速器功耗。
AI 处理器能耗已成为关键瓶颈。DVFS 是常用功耗优化手段，但 AI 处理器上控制延迟较大，已有工作通常只在程序整体或子阶段粒度调节；最新 Ascend NPU 的毫秒级 DVFS 允许对算子单独设定频率。为此，作者为每个算子构建性能与功耗模型：通过时间线分析将算子周期数建模为频率的凸分段线性函数，平均误差 1.96%；功耗模型引入温度相关项，平均误差 4.62%。基于这些模型与细粒度 DVFS，提出融合算子分类、预处理和遗传算法搜索的 DVFS 策略。在包括 GPT-3 训练的应用上，AICore 功耗降低 13.44%，NPU 芯片功耗降低 4.95%，性能损失限制在 1.76%。
Recent advancements in deep learning have significantly increased AI processors' energy consumption, which is becoming a critical factor limiting AI development. Dynamic Voltage and Frequency Scaling (DVFS) stands as a key method in power optimization. However, due to the latency of DVFS control in AI processors, previous works typically apply DVFS control at the granularity of a program's entire duration or sub-phases, rather than at the level of AI operators. The advent of millisecond-level DVFS capabilities on the latest Ascend NPU platforms enables us to set frequency individually for single or multiple operators, opening up the opportunity for further enhancing energy efficiency through fine-grained DVFS control. To ensure performance is unaffected in DVFS, our work builds performance and power models for each operator. Through in-depth timeline analysis, we demonstrate that the cycle count of an operator can be modeled as a convex piecewise linear function of frequency, resulting in a performance model with an average error of 1.96%. Moreover, we build power models that incorporate temperature-dependent terms, which enhances the model's precision and results in an average error of 4.62%. Based on our performance and power models as well as the fine-grained DVFS functionality of Ascend NPU, we propose a DVFS strategy that integrates operator classification, preprocessing, and a genetic algorithm-based search. Experiments on applications including GPT-3 training achieve a reduction in AICore (the computing component within the Ascend NPU) power by 13.44% and NPU chip power by 4.95%, while limiting performance degradation to 1.76%.
@inproceedings{WangZWWLHZXHWDC25,
title = {Using Analytical Performance/Power Model and Fine-Grained DVFS to Enhance AI Accelerator Energy Efficiency},
author = {Zibo Wang and Yijia Zhang and Fuchun Wei and Bingqiang Wang and Yanlin Liu and Zhiheng Hu and Jingyi Zhang and Xiaoxin Xu and Jian He and Xiaoliang Wang and Wanchun Dou and Guihai Chen and Chen Tian},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3669940.3707231}
}