一句话：结合人工归纳的80条语言模式与机器学习/深度学习特征，自动检测未标注的性能问题报告。
性能问题会影响软件系统的响应时间与资源消耗，但问题跟踪系统中的性能问题常缺乏显式标签，例如 Apache Jira 中性能标签率不足 1%。本文人工分析了 980 条真实性能问题报告，归纳出 80 条与项目无关、在报告中反复出现的语言模式，并以此构造句子级和问题级学习特征，训练机器学习/深度学习分类器来自动识别性能问题报告。作者在两个各含 980 条未分类报告的数据集上评估，与 31 种基线方法比较，取得最高 83% 的精确率与 59% 的召回率；唯一可比的基线方法 BERT 在 F1 分数上仍低 25%。
Performance issues compromise the response time and resource consumption of a software system. Modern software systems use issue tracking systems to manage all kinds of issue reports, including performance issues. The problem is that performance issues are often not explicitly tagged. The tagging mechanism, if exists, is completely voluntary, depending on the project’s convention and on submitters’ discipline. For example, the performance tag rate in Apache’s Jira system is below 1%. This paper contributes a hybrid classification approach that combines linguistic patterns and machine/deep learning techniques to automatically detect performance issue reports. We manually analyzed 980 real-life performance issue reports and derived 80 project-agnostic linguistic patterns that recur in the reports. Our approach uses these linguistic patterns to construct the sentence-level and issue-level learning features for training effective machine/deep learning classifiers. We test our approach on two separate datasets, each consisting of 980 unclassified issue reports, and compare the results with 31 baseline methods. Our approach can reach up to 83% precision and up to 59% recall. The only comparable baseline method is BERT, which is still 25% lower in the F1-score.
DOI 原文 ·
@inproceedings{paperbot5816,
title = {Automatically identifying performance issue reports with heuristic linguistic patterns},
author = {Yutong Zhao and Lu Xiao and Pouria Babvey and Lei Sun and Sunny Wong and Angel A. Martinez and Xiao Wang},
booktitle = {FSE},
year = {2020},
doi = {10.1145/3368089.3409674}
}