一句话：发布首个跨 NVIDIA GPU、华为 NPU 与 Google TPU 的深度学习 kernel 生成基准，含 300 个任务、15 类算子。
用大语言模型自动生成深度学习 kernel 有望减少编写高性能算子所需的人工投入与硬件专业知识，但现有评测基准存在硬件平台支持有限、kernel 分类粒度粗糙、任务覆盖不均衡等问题。本文提出 MultiKernelBench，这是首个面向 LLM 深度学习 kernel 生成的多平台综合基准，覆盖 15 个明确定义的 kernel 类别、共 300 个任务，并支持 NVIDIA GPU、华为 NPU 与 Google TPU 三大硬件平台。为实现可扩展性，作者设计了模块化后端抽象层，将平台相关逻辑与核心评测基础设施解耦，便于接入新硬件。此外，文中提出一种简单而有效的类别感知 one-shot 提示方法，通过提供同类示例提升生成质量。对七个主流 LLM 的系统评测揭示：任务难度差异显著、对训练中曝光较少的平台泛化能力差，且针对性提示策略确有成效。
The automatic generation of deep learning (DL) kernels using large language models (LLMs) has emerged as a promising approach to reduce the manual effort and hardware-specific expertise required for writing high-performance operator implementations. However, existing benchmarks for evaluating LLMs in this domain suffer from limited hardware support, coarse-grained kernel categorization, and imbalanced task coverage. To address these limitations, we introduce MultiKernelBench, the first comprehensive, multi-platform benchmark for LLM-based DL kernel generation. MultiKernelBench spans 300 tasks across 15 well-defined kernel categories and supports three major hardware platforms: NVIDIA GPUs, Huawei NPUs, and Google TPUs. To enable future extensibility, we design a modular backend abstraction layer that decouples platform-specific logic from the core benchmarking infrastructure, allowing easy integration of new hardware platforms. We further propose a simple yet effective category-aware one-shot prompting method that improves generation quality by providing in-category exemplars. Through systematic evaluations of seven state-of-the-art LLMs, we reveal significant variation in task difficulty, poor generalization to platforms with less training exposure, and the effectiveness of targeted prompting strategies.
@inproceedings{abs-2507-17773,
title = {MultiKernelBench: A Multi-platform Benchmark for Kernel Generation},
author = {Zhongzhen Wen and Yinghui Zhang and Zhong Li and Zhongxin Liu and Linna Xie and Tian Zhang},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3837460}
}