一句话：提出基于局部规则的模型无关解释技术 PyExplainer，为即时缺陷预测模型生成更准确、一致且独特的解释。
即时（JIT）缺陷预测用 AI/ML 判断引入缺陷的提交，帮助开发者优先分配 SQA 资源，但模型缺乏可解释性。LIME 虽可用于解释，但其随机扰动生成的合成近邻与待解释实例不够相似，导致局部模型精度低、解释不准。本文提出 PyExplainer，一种基于局部规则的模型无关解释技术，用于解释 JIT 缺陷模型为何将某提交预测为缺陷引入。在两个开源项目案例研究中，PyExplainer 生成的合成近邻相似度比 LIME 高 41%-45%，局部模型精度高 18%-38%，解释独特性高 69%-98%，与未来实际缺陷引入提交特征的一致性高 17%-54%。作者发布 Python 包以支持实践。
Just-In-Time (JIT) defect prediction (i.e., an AI/ML model to predict defect-introducing commits) is proposed to help developers prioritize their limited Software Quality Assurance (SQA) resources on the most risky commits. However, the explainability of JIT defect models remains largely unexplored (i.e., practitioners still do not know why a commit is predicted as defect-introducing). Recently, LIME has been used to generate explanations for any AI/ML models. However, the random perturbation approach used by LIME to generate synthetic neighbors is still suboptimal, i.e., generating synthetic neighbors that may not be similar to an instance to be explained, producing low accuracy of the local models, leading to inaccurate explanations for just-in-time defect models. In this paper, we propose PyExplainer—i.e., a local rule-based model-agnostic technique for generating explanations (i.e., why a commit is predicted as defective) of JIT defect models. Through a case study of two open-source software projects, we find that our PyExplainer produces (1) synthetic neighbors that are 41%-45% more similar to an instance to be explained; (2) 18%-38% more accurate local models; and (3) explanations that are 69%-98% more unique and 17%-54% more consistent with the actual characteristics of defect-introducing commits in the future than LIME (a state-ofthe-art model-agnostic technique). This could help practitioners focus on the most important aspects of the commits to mitigate the risk of being defect-introducing. Thus, the contributions of this paper build an important step towards Explainable AI for Software Engineering, making software analytics more explainable and actionable. Finally, we publish our PyExplainer as a Python package to support practitioners and researchers (https://github.com/awsm-research/PyExplainer).
DOI 原文 ·
@inproceedings{paperbot6529,
title = {PyExplainer: Explaining the Predictions of Just-In-Time Defect Models},
author = {Chanathip Pornprasit and Chakkrit Tantithamthavorn and Jirayus Jiarpakdee and Michael Fu and Patanamon Thongtanunam},
booktitle = {ASE},
year = {2021},
doi = {10.1109/ase51524.2021.9678763}
}