一句话：实证评估现有 API 推荐方法在 Python 机器学习任务上的不足，并提出基于频繁项集挖掘的 FIMAX 方法显著提升推荐效果。
API 推荐能帮助开发者更高效地构建软件，但现有方法主要在 Java 等静态类型语言的通用编程任务上评估，对 Python 机器学习（ML）任务的实用性和有效性尚不清楚。本文针对 Stack Overflow 上 Python ML 编程任务，选取六个常用 ML 库和两种先进 API 推荐方法 BIKER 与 DeepAPI 开展实证研究，发现现有方法表现不佳，主要原因是 ML 任务常需要较长的 API 序列，且存在现有方法无法处理的常见 API 使用模式。受此启发，作者提出简单而有效的频繁项集挖掘方法 FIMAX，利用 Stack Overflow 问题中的常见 API 使用信息增强现有推荐方法。评估表明，FIMAX 在 MRR 和 MAP 上分别最高提升 54.3% 和 57.4%，14 名开发者参与的用户研究进一步验证了其实用性。
Application Programming Interfaces (APIs) are designed to help developers build software more effectively. Recommending the right APIs for specific tasks is gaining increasing attention among researchers and developers. However, most of the existing approaches are mainly evaluated for general programming tasks using statically typed programming languages such as Java. Little is known about their practical effectiveness and usefulness for machine learning (ML) programming tasks with dynamically typed programming languages such as Python, whose paradigms are fundamentally different from general programming tasks. This is of great value considering the increasing popularity of ML and the large number of new questions appearing on question answering websites. In this work, we set out to investigate the effectiveness of existing API recommendation approaches for Python-based ML programming tasks from Stack Overflow (SO). Specifically, we conducted an empirical study of six widely-used Python-based ML libraries using two state-of-the-art API recommendation approaches, i.e., BIKER and DeepAPI. We found that the existing approaches perform poorly for two main reasons: (1) Python-based ML tasks often require significant long API sequences; and (2) there are common API usage patterns in Python-based ML programming tasks that existing approaches cannot handle. Inspired by our findings, we proposed a simple but effective frequent itemset mining-based approach, i.e., FIMAX, to boost API recommendation approaches, i.e., enhance existing API recommendation approaches for Python-based ML programming tasks by leveraging the common API usage information from SO questions. Our evaluation shows that FIMAX improves existing state-of-the-art API recommendation approaches by up to 54.3% and 57.4% in MRR and MAP, respectively. Our user study with 14 developers further demonstrates the practicality of FIMAX for API recommendation.
DOI 原文 ·
@inproceedings{paperbot7114,
title = {API recommendation for machine learning libraries: how far are we?},
author = {Moshi Wei and Yuchao Huang and Junjie Wang and Jiho Shin and Nima Shiri Harzevili and Song Wang},
booktitle = {FSE},
year = {2022},
doi = {10.1145/3540250.3549124}
}