一句话：首次系统实证研究自动化参数优化对62种跨项目缺陷预测技术的影响，发现其显著提升性能且迁移学习参数最关键。
跨项目缺陷预测（CPDP）通常由迁移学习和分类两部分构成，每部分都含需调优的超参数，但自动化参数优化对其影响缺乏系统认识。本文构建并评估62种CPDP技术，其中13种来自已有文献，49种为新组合，在20个不同规模和特征的真实软件项目上建立缺陷预测模型进行大规模实证。结果显示：自动化参数优化能以可控计算开销显著提升77%技术的预测性能；迁移学习至关重要，预算紧张时优先优化迁移学习参数更具性价比；通过组合现有迁移学习与分类技术即可较容易找到更优替代方案，说明CPDP研究仍不成熟。
Data-driven defect prediction has become increasingly important in software engineering process. Since it is not uncommon that data from a software project is insufficient for training a reliable defect prediction model, transfer learning that borrows data/konwledge from other projects to facilitate the model building at the current project, namely cross-project defect prediction (CPDP), is naturally plausible. Most CPDP techniques involve two major steps, i.e., transfer learning and classification, each of which has at least one parameter to be tuned to achieve their optimal performance. This practice fits well with the purpose of automated parameter optimization. However, there is a lack of thorough understanding about what are the impacts of automated parameter optimization on various CPDP techniques. In this paper, we present the first empirical study that looks into such impacts on 62 CPDP techniques, 13 of which are chosen from the existing CPDP literature while the other 49 ones have not been explored before. We build defect prediction models over 20 real-world software projects that are of different scales and characteristics. Our findings demonstrate that: (1) Automated parameter optimization substantially improves the defect prediction performance of 77% CPDP techniques with a manageable computational cost. Thus more efforts on this aspect are required in future CPDP studies. (2) Transfer learning is of ultimate importance in CPDP. Given a tight computational budget, it is more cost-effective to focus on optimizing the parameter configuration of transfer learning algorithms (3) The research on CPDP is far from mature where it is 'not difficult' to find a better alternative by making a combination of existing transfer learning and classification techniques. This finding provides important insights about the future design of CPDP techniques.
DOI 原文 ·
@inproceedings{paperbot5653,
title = {Understanding the automated parameter optimization on transfer learning for cross-project defect prediction},
author = {Ke Li and Zilin Xiang and Tao Chen and Shuo Wang and Kay Chen Tan},
booktitle = {ICSE},
year = {2020},
doi = {10.1145/3377811.3380360}
}