一句话：通过跨三代 GPU 的纵向研究，构建 CaSiO 应用套件，揭示深度学习软硬件协同演进趋势。
MLPerf 等基准以 CNN 为主，难以反映 AI 用例的多样性，也只覆盖生产 AI 的一小部分。本文对视觉、物理模拟、视觉合成、语言与语音处理、表格数据处理等前沿 AI 应用开展纵向研究，覆盖三代硬件，并将这些应用及执行脚手架称为 CaSiO 套件。作者在框架层、设备 API 层以及硬件/微架构层采集数据，分析 AI 革命中软硬件共同演进的过程，并给出未来趋势的判断。该工作为理解 GPU 上深度学习负载、框架与硬件之间的相互作用提供了跨代际的实证视角。
We are in age of AI, with rapidly changing algorithms and a somewhat synergistic change in hardware. MLPerf is a recent benchmark suite that serves as a way to compare and evaluate hardware. However it has several drawbacks - it is dominated by CNNs and does a poor job of capturing the diversity of AI use cases, and only represents a sliver of production AI use cases. This paper performs a longitudinal study of state-of-art AI applications spanning vision, physical simulation, vision synthesis, language and speech processing, and tabular data processing, across three generations of hardware to understand how the AI revolution has panned out. We call this collection of applications and execution scaffolding the CaSiO suite. The paper reports on data gathered at the framework level, device API level, and hardware and microarchitecture level. The paper provides insights on the hardware-software revolution with pointers to future trends.
@inproceedings{paperbot11244,
title = {A Journey of a 1,000 Kernels Begins with a Single Step: A Retrospective of Deep Learning on GPUs},
author = {Michael Davies and Ian McDougall and Selvaraj Anandaraj and Deep Machchhar and Rithik Jain and Karthikeyan Sankaralingam},
booktitle = {ASPLOS},
year = {2024},
doi = {10.1145/3620665.3640367}
}