一句话:提出 CLAA,用监督对比学习训练 Transformer,显著提升 API 评论中性能、安全、可用性、文档等关注点的检测效果。
论文针对 API 评论中自动识别开发者关注点(如性能、安全、可用性、文档)的任务,提出 CLAA 方法:以 Transformer 模型为基础,采用监督对比学习目标进行训练,使同一方面的表示更聚合、不同方面更可分。评估包括性能与影响分析:在 Stack Overflow 开发者讨论基准数据集上与 SOTA Transformer 比较,CLAA 在多个方面检测上显著提升;在随机抽取并人工标注的 200 条在线评论上达到 92% 准确率,而 SOTA 基线为 81.5%。10 名开发者参与的研究显示,Stack Overflow+CLAA 能提高 API 选择时的准确性与信心。
We present a novel approach - CLAA - for API aspect detection in API reviews that utilizes transformer models trained with a supervised contrastive loss objective function. We evaluate CLAA using performance and impact analysis. For performance analysis, we utilized a benchmark dataset on developer discussions collected from Stack Overflow and compare the results to those obtained using state-of-the-art transformer models. Our experiments show that contrastive learning can significantly improve the performance of transformer models in detecting aspects such as Performance, Security, Usability, and Documentation. For impact analysis, we performed empirical and developer study. On a randomly selected and manually labeled 200 online reviews, CLAA achieved 92% accuracy while the SOTA baseline achieved 81.5%. According to our developer study involving 10 participants, the use of Stack Overflow + CLAA resulted in increased accuracy and confidence during API selection. Replication package: https://github.com/disa-lab/Contrastive-Learning-API-Aspect-ASE2023.