一句话:比较传统机器学习与深度学习方法在自动识别 API 参考文档所含知识类型上的表现,单类型分类最佳 AUPRC 达 87%。
API 参考文档(如 JavaDoc)是软件开发的重要组成部分,已有工作提出包含 12 种知识类型(如功能、结构与质量)的 grounded taxonomy,但自动识别文档中的特定知识类型仍具挑战。本文研究现代文本分类方法能否有效完成该任务。作者基于人工标注的 Java 与 .NET API 文档(n = 5,574)构建数据集,对比传统机器学习(k-NN、SVM)与深度学习方法,并在单类型(多个二元分类器)与多类型设置下评估。实验表明,单类型分类时最佳 AUPRC 高达 87%,说明现代文本分类方法能够较准确地识别 API 文档中的知识类型。
Using API reference documentation like JavaDoc is an integral part of software development. Previous research introduced a grounded taxonomy that organizes API documentation knowledge in 12 types, including knowledge about the Functionality, Structure, and Quality of an API. We study how well modern text classification approaches can automatically identify documentation containing specific knowledge types. We compared conventional machine learning (k-NN and SVM) with deep learning approaches trained on manually-annotated Java and .NET API documentation (n = 5,574). When classifying the knowledge types individually (i.e., multiple binary classifiers) the best AUPRC was up to 87