一句话：提出首个评估大语言模型 SQL 方言翻译能力的综合基准 DLBench，包含两个经过严格清洗与校验的数据集。
数据库管理系统日益复杂、SQL 方言不断增多，给数据库迁移、联邦与集成带来显著困难，跨方言的 SQL 翻译因此十分关键。本文提出 DLBench，这是首个用于评估大语言模型 SQL 翻译能力的综合基准，包含两个数据集：覆盖七种 DBMS 真实查询场景的 BIRDTrans，以及涵盖更广泛 SQL 类型与丰富方言特性的 ButterTrans。作者收集高质量数据库与 SQL 语句，并通过多步清洗流程，结合基于 SQL92 的检查与方言专用解析器验证来保证数据质量，同时使用基于 LLM 与人工的标注确保数据集的正确性与完整性。大量实验表明该基准能有效评估 LLM 的 SQL 翻译能力，展示了其潜力并指出了后续改进方向。
In recent years, the growing complexity of database management systems (DBMSs) and the proliferation of SQL dialects have created significant challenges for database migration, federation, and integration. These challenges arise from the disparities between SQL dialects across different DBMSs, hindering seamless communication and system interoperability. SQL translation, the process of converting SQL queries from a source dialect DBMS to a target dialect DBMS, plays a crucial role in addressing these challenges. To facilitate this process, we introduce DLBench, the first comprehensive benchmark designed to evaluate the SQL translation capabilities of Large Language Models (LLMs). The benchmark includes two datasets: BIRDTrans, which covers real-world database query scenarios across seven DBMSs, and ButterTrans, which spans a broader spectrum of SQL types and encompasses extensive DBMS dialect features. We collect high-quality databases and SQL statements, applying a rigorous multi-step cleaning process that ensures data quality through SQL92-based checks and dialect-specific parser validation. Additionally, both LLM-based and human annotations are used to guarantee the correctness and completeness of the dataset. We demonstrate the utility of DLBench through extensive experiments, which show that the benchmark effectively evaluates the SQL translation ability of LLMs. The results highlight the potential of LLMs for SQL translation tasks and provide insights into areas for further improvement.
DOI 原文 ·
@inproceedings{paperbot9447,
title = {DLBench: A Comprehensive Benchmark for SQL Translation with Large Language Models},
author = {Li Lin and Hongqiao Chen and Qinglin Zhu and Liehang Chen and Linlong Tang and Rongxin Wu},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00076}
}