一句话：HARP 通过跨计算图与宿主 Python 代码的整体分析来重构 ML 程序，平均加速 2.07 倍。
现代机器学习程序多以 Python 编写，其核心计算通过调用 TensorFlow、PyTorch 等高度优化的库实现。现有优化工作主要关注库 API 所指定的静态计算图，而基本忽略了宿主 Python 代码的影响，因而难以把握全局、错失大量优化机会。本文提出 HARP，实现跨越计算图与其宿主 Python 代码的整体分析：通过面向分析的推测性分析绕开 Python 的复杂语言特性，用增强计算图这一统一表示刻画整体分析所需的各维度知识，并以条件化反馈机制支持风险可控的激进分析。基于 HARP 的程序重构在一组 TensorFlow 与 PyTorch 程序上取得 1.3–3 倍加速，平均加速 2.07 倍。
Modern machine learning programs are often written in Python, with the main computations specified through calls to some highly optimized libraries (e.g., TensorFlow, PyTorch). How to maximize the computing efficiency of such programs is essential for many application domains, which has drawn lots of recent attention. This work points out a common limitation in existing efforts: they focus their views only on the static computation graphs specified by library APIs, but leave the influence from the hosting Python code largely unconsidered. The limitation often causes them to miss the big picture and hence many important optimization opportunities. This work proposes a new approach named HARP to address the problem. HARP enables holistic analysis that spans across computation graphs and their hosting Python code. HARP achieves it through a set of novel techniques: analytics-conscious speculative analysis to circumvent Python complexities, a unified representation augmented computation graphs to capture all dimensions of knowledge related with the holistic analysis, and conditioned feedback mechanism to allow risk-controlled aggressive analysis. Refactoring based on HARP gives 1.3--3X and 2.07X average speedups on a set of TensorFlow and PyTorch programs.
DOI 原文 ·
@inproceedings{paperbot5666,
title = {HARP},
author = {Weijie Zhou and Yue Zhao and Guoqiang Zhang and Xipeng Shen},
booktitle = {ICSE},
year = {2020},
doi = {10.1145/3377811.3380434}
}