一句话：提出基于反例引导合成与列式分解的 RDD2SQL，将 Spark 函数式查询自动转译为 SQL 并提升性能。
Apache Spark、Flink 等大数据框架允许用户用函数式 API 编写分布式查询，但这类查询通常不如 SQL 高效。本文提出一种将函数式查询自动转译为 SQL 的技术。方法基于反例引导的归纳合成（CEGIS），并引入新颖的按列分解技术，将整体合成任务拆分为若干更小的子查询合成问题。作者实现工具 RDD2SQL，用于把 Spark RDD 查询翻译为 SQL，并在真实 RDD 查询上评估。结果表明：大多数 RDD 查询可被翻译为 SQL，工具能有效自动化该翻译，且翻译后获得显著性能收益。
Big data analytics frameworks like Apache Spark and Flink enable users to implement queries over large, distributed databases using functional APIs. In recent years, these APIs have grown in popularity because their functional interfaces abstract away much of the minutiae of distributed programming required by traditional query languages like SQL. However, the convenience of these APIs comes at a cost because functional queries are often less efficient than their SQL counterparts. Motivated by this observation, we present a new technique for automatically transpiling functional queries to SQL. While our approach is based on the standard paradigm of counterexample-guided inductive synthesis, it uses a novel column-wise decomposition technique to split the synthesis task into smaller subquery synthesis problems. We have implemented this approach as a new tool called RDD2SQL for translating Spark RDD queries to SQL and empirically evaluate the effectiveness of RDD2SQL on a set of real-world RDD queries. Our results show that (1) most RDD queries can be translated to SQL, (2) our tool is very effective at automating this translation, and (3) performing this translation offers significant performance benefits.
@article{paperbot2022,
title = {Automated Translation of Functional Big Data Queries to SQL},
author = {Guoqiang Zhang and Benjamin Mariano and Xipeng Shen and Işıl Dillig},
journal = {Proceedings of the ACM on Programming Languages},
volume = {7},
number = {OOPSLA1},
year = {2023},
doi = {10.1145/3586047}
}