一句话：Virgo 将矩阵单元从 SIMT 核解耦并集成到核集群层级，提升 GPU 矩阵运算可扩展性与能效。
现代GPU用Tensor Core等矩阵单元加速深度学习中的GEMM，但现有设计紧耦合SIMT核，受寄存器文件容量与带宽限制，操作规模难扩展，吞吐与能效难以同步提升。Virgo将矩阵单元集成到SIMT核集群层级，通过解耦消除核微架构约束；提高硬件操作粒度以减少指令处理能耗；统一矩阵单元设计并将操作数与累加器访问从寄存器文件卸载，提升数据复用与能效，还支持SIMT核与矩阵单元并发执行以优化融合DNN工作负载。基于可综合RTL的评估表明，相比Ampere风格与Hopper风格核耦合设计，其片上动态功耗分别降低67.3%和24.2%。
Modern GPUs incorporate specialized matrix units such as Tensor Cores to accelerate GEMM operations, which are central to deep learning workloads. However, existing matrix unit designs are tightly coupled to the SIMT core, restricting operation size due to register file capacity and bandwidth constraints. Such a limitation in scalability makes it difficult to simultaneously improve compute throughput and energy efficiency in GPUs. To address this challenge, we propose Virgo, a GPU microarchitecture that integrates dedicated matrix units at the SIMT core cluster level. By decoupling the matrix unit from the SIMT core, Virgo eliminates scalability constraints imposed by the core microarchitecture. Consequently, Virgo increases operation granularity at the hardware level, reducing energy overhead from core instruction processing. Physical disaggregation also enables a unified matrix unit design and offloading both operand and accumulator accesses from the register file, improving data reuse and energy efficiency. Furthermore, this disaggregation supports efficient concurrent execution of the SIMT core and matrix unit, optimizing mapping for fused DNN workloads. Our evaluations using synthesizable RTL demonstrate that Virgo achieves 67.3% and 24.2% reduction in on-chip active power consumption, compared to the baseline Ampere-style and Hopper-style core-coupled designs.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2408-12073,
title = {Virgo: Cluster-level Matrix Unit Integration in GPUs for Scalability and Energy Efficiency},
author = {Hansung Kim and Ruohan Richard Yan and Joshua You and Tieliang Vamber Yang and Yakun Sophia Shao},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3676641.3716281}
}