一句话：提出一种轻量级DNN测试选择方法，利用测试样本自身及其邻居的不确定性，增强简单不确定性选择方法的效果。
深度神经网络的脆弱性限制其在高风险场景中的部署，测试用例选择或优先级排序可缓解深度学习软件故障风险。现有基于不确定性的简单方法（如DeepGini）在发现DNN故障方面有效，而学习式TestRank虽表现更优，却需训练图卷积网络和多层感知机，设计复杂。本文提出一种新的轻量级DNN测试选择方法：除测试样本自身的不确定性外，还考虑模型在其邻居上的不确定性，从而增加所选测试用例的多样性，提升现有不确定性选择方法的有效性。在5个数据集上的大量实验验证了该方法的有效性。
Although deep learning (DL) software has been pervasive in various applications, the brittleness of deep neural networks (DNN) hinders their deployment in many tasks especially high-stake ones. To mitigate the risk accompanied with DL software fault, a variety of DNN testing techniques have been proposed such as test case selection. Among those test case selection or prioritization methods, the uncertainty-based ones such as DeepGini have demonstrated their effectiveness in finding DNN’s faults. Recently, TestRank, a learning based test ranking method has shown their out-performance over simple uncertainty-based test selection methods. However, this is achieved with a more complicated design which needs to train a graph convolutional network and a multi-layer Perceptron. In this paper, we propose a novel and lightweight DNN test selection method to enhance the effectiveness of existing simple ones. Besides the DNN model’s uncertainty on test case itself, we take into account model’s uncertainty on its neighbors. This could diversify the selected test cases and improve the effectiveness of existing uncertainty-based test selection methods. Extensive experiments on 5 datasets demonstrate the effectiveness of our approach.
DOI 原文 ·
@inproceedings{paperbot8066,
title = {In Defense of Simple Techniques for Neural Network Test Case Selection},
author = {Shenglin Bao and Chaofeng Sha and Bihuan Chen and Xin Peng and Wenyun Zhao},
booktitle = {ISSTA},
year = {2023},
doi = {10.1145/3597926.3598073}
}