一句话:提出无需人工标注的 ARCLIN,从论坛非结构化文本中识别并链接 API 提及,在 Py-mention 上较启发式方法提升 8%。
在线技术论坛中蕴含大量官方文档未覆盖的 API 使用知识,但论坛文本短、非正式且含拼写错误和缩写,自动识别并链接 API 提及面临三个挑战:区分 API 与普通词、识别非全限定名、消歧不同库中同名方法。本文提出 ARCLIN,无需人工标注:先用 Bi-LSTM 之上的 CRF 从自然语言句中抽取 API 提及,再用上下文感知打分机制计算提及与 API 仓库条目的相似度。在包含五个 Python 库、558 个提及和 2,830 句的 Py-mention 数据集上,ARCLIN 比启发式方法提升 8%,是首个无需人工标签即可全自动完成非格式化文本 API 提及解析的方法。
Online technical forums (e.g., StackOverflow) are popular platforms for developers to discuss technical problems such as how to use a specific Application Programming Interface (API), how to solve the programming tasks, or how to fix bugs in their code. These discussions can often provide auxiliary knowledge of how to use the software that is not covered by the official documents. The automatic extraction of such knowledge may support a set of down-stream tasks like API searching or indexing. However, unlike official documentation written by experts, discussions in open forums are made by regular developers who write in short and informal texts, including spelling errors or abbreviations. There are three major challenges for the accurate APIs recognition and linking mentioned APIs from unstructured natural language documents to an entry in the API repository: (1) distinguishing API mentions from common words; (2) identifying API mentions without a fully qualified name; and (3) disambiguating API mentions with similar method names but in a different library. In this paper, to tackle these challenges, we propose an ARCLIN tool, which can effectively distinguish and link APIs without using human annotations. Specifically, we first design an API recognizer to automatically extract API mentions from natural language sentences by a Conditional Random Field (CRF) on the top of a Bi-directional Long Short-Term Memory (Bi-LSTM) module, then we apply a context-aware scoring mechanism to compute the mention-entry similarity for each entry in an API repository. Compared to previous approaches with heuristic rules, our proposed tool without manual inspection outperforms by 8% in a high-quality dataset Py-mention, which contains 558 mentions and 2,830 sentences from five popular Python libraries. To our best knowledge, ARCLIN is the first approach to achieve full automation of API mention resolution from unformatted text without manually collected labels.