一句话：提出 WiseSword，自动将 DLRM 中 VC 密集型特征处理分支卸载至 CPU，实现 CPU-NPU 流水线推理并提升吞吐。
Ascend NPU 包含负责轻量计算的向量核（VC）和负责矩阵乘的立方核（CC）。在 DLRM 中，大量轻量算子使 VC 过载而 CC 利用不足，导致执行不均衡和推理吞吐下降。作者观察到许多 VC 密集型算子位于独立的特征处理分支，可卸载到 CPU，从而形成 CPU-NPU 流水线推理。为在多样模型架构与资源规格下满足工业生产的请求尾延迟和 CPU 利用率约束，论文提出 WiseSword：一种面向高吞吐 DLRM 部署的自动计算图划分工具。它采用 light-first 原则指导的渐进式子图搜索，依据批大小可压缩性和计算开销逐步卸载特征处理分支，直到 CPU 利用率约束被违反或吞吐下降，从而充分利用可用 CPU 资源；并利用并发调优满足尾延迟约束。生产硬件实验表明，WiseSword 在严格满足 CPU 利用率和尾延迟约束的同时显著提升吞吐，并已部署于华为生产环境。
Ascend neural processing units (NPUs) integrate vector cores (VCs) for compute-light operations and cube cores (CCs) for matrix multiplications. In deep learning recommendation models (DLRMs), lightweight operators overload VCs, while leaving CCs underutilized, resulting in severe execution imbalance and degraded inference throughput. We observe that many VC-intensive operators reside in independent feature-processing branches, which can be offloaded to CPUs to enable CPU-NPU pipelined inference. However, adaptively optimizing partition schemes for diverse model architectures and resource specifications remains a significant challenge, especially under request tail latency and CPU utilization constraints in industrial production. To address this challenge, we propose WiseSword, an automated computation graph partitioning tool for high-throughput DLRM deployment. WiseSword employs a progressive subgraph search guided by the light-first principle, which incrementally offloads feature-processing branches based on their batch-size compressibility and computation overhead until the CPU utilization constraint is violated or throughput degrades, thus fully exploiting available CPU resources. Furthermore, it leverages concurrency tuning to satisfy the tail-latency constraint. Experiments on production hardware confirm that WiseSword greatly improves throughput while strictly adhering to both CPU utilization and tail latency constraints. Currently, WiseSword has been deployed in Huawei’s production environment.
@inproceedings{paperbot12330,
title = {WiseSword: An Automated Computation Graph Partitioning Tool for Industrial DLRM Deployment on Ascend NPUs},
author = {Ke Cheng and Yuhan Zhang and Bo Pan and Jiawei Zhu and Qidong Zhang and Yuting Yan and Sheng Zhang and Yibo Jin and Zhengyong Zhang},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834464}
}