一句话：将通用 Python 函数转换为程序生成器，执行时生成静态类型 IR，实现高层领域优化与高效 C++ 代码生成。
数据科学工作负载常包含 Python 代码，但 Python 的动态特性使其难以高效执行。传统方法或将 Python 视为黑盒，错失优化机会，或局限于狭窄领域。本文提出一种新方法：将通用 Python 函数转换为程序生成器，在执行时生成静态类型 IR，从而支持高层、领域特定的优化并生成高效 C++ 代码。原型系统 HiPy 在多种工作负载上实现单线程 2–20 倍加速，在数值数据等其他领域也能加速 Python 代码，有时甚至超越专用编译器。
Data science workloads frequently include Python code, but Python’s dynamic nature makes efficient execution hard. Traditional approaches either treat Python as a black box, missing out on optimization potential, or are limited to a narrow domain. However, a deep and efficient integration of user-defined Python code into data processing systems requires extracting the semantics of the entire Python code. In this paper, we propose a novel approach for extracting the high-level semantics by transforming general Python functions into program generators that generate a statically-typed IR when executed. The extracted IR then allows for high-level, domain-specific optimizations and the generation of efficient C++ code. With our prototype implementation, HiPy, we achieve single-threaded speedups of 2–20x for many workloads. Furthermore, HiPy is also capable of accelerating Python code in other domains like numerical data, where it can sometimes even outperform specialized compilers.
@article{JungmairEG24,
title = {HiPy: Extracting High-Level Semantics from Python Code for Data Processing},
author = {Michael Jungmair and Alexis Engelke and Jana Giceva},
journal = {Proceedings of the ACM on Programming Languages},
volume = {8},
number = {OOPSLA2},
year = {2024},
doi = {10.1145/3689737}
}