一句话：提出 DATIS，利用测试输入与最近邻训练样本及彼此距离选择高故障揭示力的 DNN 测试输入。
DNN 测试通常依赖大量带标签的测试输入，获取成本高且资源密集。本文提出 DATIS，一种距离感知的测试输入选择方法：先根据测试输入到最近邻训练样本的距离改进不确定性分数并据此筛选，再通过已选测试输入之间的距离去除可能覆盖相同故障的冗余样本。作者在 8 个不同领域、结构和输入类型的对象上开展实验，与 15 个基线方法比较，结果表明 DATIS 在高故障揭示力测试输入选择及指导 DNN 增强数据选择方面均显著更优。
Deep Neural Network (DNN) testing is one of the common practices to guarantee the quality of DNNs. However, DNN testing in general requires a significant amount of test inputs with oracle information (labels), which can be challenging and resource-intensive to obtain. To relieve this problem, we propose DATIS, a distance-aware test input selection approach for DNNs. Specifically, DATIS adopts a two-step approach for selecting test inputs. In the first step, it selects test inputs based on improved uncertainty scores derived from the distances between the test inputs and their nearest neighbor training samples. In the second step, it further eliminates test inputs that may cover the same faults by examining the distances among the selected test inputs. To evaluate DATIS, we conduct extensive experiments on 8 diverse subjects, taking into account different domains of test inputs, varied DNN structures, and diverse types of test inputs. Evaluation results show that DATIS significantly outperforms 15 baseline approaches in both selecting test inputs with high fault-revealing power and guiding the selection of data for DNN enhancement.
DOI 原文 ·
@inproceedings{paperbot8497,
title = {Distance-Aware Test Input Selection for Deep Neural Networks},
author = {Zhong Li and Zhengfeng Xu and Ruihua Ji and Minxue Pan and Tian Zhang and Linzhang Wang and Xuandong Li},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3652125}
}