一句话：提出 QuanBench 基准，用 44 个量子编程任务评测 LLM 的量子代码生成能力，发现正确率不足 40%。
大语言模型在通用代码生成上表现良好，但其量子代码生成能力尚缺乏系统研究。本文提出 QuanBench，一个用于评测 LLM 量子代码生成能力的基准，包含 44 个编程任务，覆盖量子算法、态制备、门分解与量子机器学习。每个任务都配有可执行的规范解答，并从功能正确性（Pass@K）与量子语义等价性（Process Fidelity）两个维度进行评估。作者评测了若干近期模型，包括通用模型与代码专用模型。结果显示，当前 LLM 生成正确量子代码的能力有限，总体准确率低于 40%，且频繁出现语义错误。论文还分析了常见失败案例，如使用过时 API、电路构造错误以及算法逻辑错误。QuanBench 为未来改进 LLM 量子代码生成研究提供了基础。
Large language models (LLMs) have demonstrated good performance in general code generation; however, their capabilities in quantum code generation remain insufficiently studied. This paper presents QuanBench, a benchmark for evaluating LLMs on quantum code generation. QuanBench includes 44 programming tasks that cover quantum algorithms, state preparation, gate decomposition, and quantum machine learning. Each task has an executable canonical solution and is evaluated by functional correctness (Pass@K) and quantum semantic equivalence (Process Fidelity). We evaluate several recent LLMs, including general-purpose and code-specialized models. The results show that current LLMs have limited capability in generating the correct quantum code, with overall accuracy below 40% and frequent semantic errors. We also analyze common failure cases, such as outdated API usage, circuit construction errors, and incorrect algorithm logic. QuanBench provides a basis for future work on improving quantum code generation with LLMs.
DOI 原文 ·
@inproceedings{paperbot9536,
title = {QuanBench: Benchmarking Quantum Code Generation with Large Language Models},
author = {Xiaoyu Guo and Minggu Wang and Jianjun Zhao},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00218}
}