一句话:提出符号执行驱动方法,抽取带分支或循环的 Spark 应用可能的多条并行执行计划,替代唯一计划假设。
Spark 应用的执行顺序与并行度由作业间依赖决定,Spark 将这些依赖具体化为并行执行计划图。已有执行时间估计和动态资源供给工作通常假设:给定计算资源,应用的执行计划唯一。本文指出,该假设对含条件分支或循环的应用过于简化,会限制预测精度,因为不同数据可能触发不同执行计划。为此,论文提出以符号执行驱动的方法抽取 Spark 应用的并行执行计划,显式获得可能的多条计划而非单一计划。摘要未报告具体评估结果与实验数据。
The execution of Spark applications is based on the execution order and parallelism of the different jobs, given data and available resources. Spark reifies these dependencies in a graph that we refer to as the (parallel) execution plan of the application. All the approaches that have studied the estimation of the execution times and the dynamic provisioning of resources for this kind of applications have always assumed that the execution plan is unique, given the computing resources at hand. This assumption is at least simplistic for applications that include conditional branches or loops and limits the precision of the prediction techniques.
@inproceedings{paperbot5073,
title = {Symbolic execution-driven extraction of the parallel execution plans of Spark applications},
author = {Luciano Baresi and Giovanni Denaro and Giovanni Quattrocchi},
booktitle = {FSE},
year = {2019},
doi = {10.1145/3338906.3338973}
}