一句话：提出 RELANCER，用运行时错误驱动与机器学习自动升级废弃 API，近实时恢复 Jupyter Notebook 可执行性。
公开的计算笔记本（Jupyter Notebook）常因数据科学库快速演化导致 API 废弃而无法开箱执行。本文提出 RELANCER，以近实时方式自动升级废弃 API 来恢复 notebook 可执行性。它采用迭代的运行时错误驱动策略，每次只定位并修复一个 API 问题：先用机器学习模型根据错误消息预测修复类型（API/包名、参数或参数值），再结合 GitHub 上的 API 迁移示例与 API 文档构造候选修复空间，并用第二个机器学习模型对候选映射排序。在 Kaggle 精选的 255 个不可执行 notebook 上，RELANCER 成功恢复 56% 的可执行性，而仅依赖 GitHub 示例和仅依赖文档的基线分别只能修复 38% 与 36%；在 5 分钟实时限制内，它仍能恢复 49%，缺乏机器学习模型的基线仅 24%。
Data scientists typically practice exploratory programming using computational notebooks, to comprehend new data and extract insights. To do this they iteratively refine their code, actively trying to re-use and re-purpose solutions created by other data scientists, in real time. However, recent studies have shown that a vast majority of publicly available notebooks cannot be executed out of the box. One of the prominent reasons is the deprecation of data science APIs used in such notebooks, due to the rapid evolution of data science libraries. In this work we propose RELANCER, an automatic technique that restores the executability of broken Jupyter Notebooks, in near real time, by upgrading deprecated APIs. RELANCER employs an iterative runtime-error-driven approach to identify and fix one API issue at a time. This is supported by a machine-learned model which uses the runtime error message to predict the kind of API repair needed - an update in the API or package name, a parameter, or a parameter value. Then RELANCER creates a search space of candidate repairs by combining knowledge from API migration examples on GitHub as well as the API documentation and employs a second machine-learned model to rank this space of candidate mappings. An evaluation of RELANCER on a curated dataset of 255 un-executable Jupyter Notebooks from Kaggle shows that RELANCER can successfully restore the executability of 56% of the subjects, while baselines relying on just GitHub examples and just API documentation can only fix 38% and 36% of the subjects respectively. Further, pursuant to its real-time use case, RELANCER can restore execution to 49% of subjects, within a 5 minute time limit, while a baseline lacking its machine learning models can only fix 24%.
DOI 原文 ·
@inproceedings{paperbot6595,
title = {Restoring the Executability of Jupyter Notebooks by Automatic Upgrade of Deprecated APIs},
author = {Chenguang Zhu and Ripon K. Saha and Mukul R. Prasad and Sarfraz Khurshid},
booktitle = {ASE},
year = {2021},
doi = {10.1109/ase51524.2021.9678889}
}