一句话:提出基于决策的蜕变测试,暴露 DNN 依赖错误决策却偶然保持输出一致的隐藏缺陷。
现有 DNN 测试常采用蜕变测试(MT),通过语义保持变异输入图像并检查预测一致性。然而,DNN 可能依赖错误的决策(即输入上的某些组件)进行预测,却因偶然因素仍保持输出不变;这类缺陷会被现有 MT 框架忽略。更严重的是,错误决策可能在真实场景中多样图像上导致连续误预测。本文提出基于决策的蜕变测试,不再仅检查输出一致性,而是针对 DNN 决策所依赖的组件施加语义保持变异,以暴露隐藏缺陷。摘要未给出具体实验细节,但该工作指出现有框架的盲区,并论证基于决策的测试对发现此类 DNN 缺陷的必要性。
Contemporary DNN testing works are frequently conducted using metamorphic testing (MT). In general, de facto MT frameworks mutate DNN input images using semantics-preserving mutations and determine if DNNs can yield consistent predictions. Nevertheless, we find that DNNs may rely on erroneous decisions (certain components on the DNN inputs) to make predictions, which may still retain the outputs by chance. Such DNN defects would be neglected by existing MT frameworks. Erroneous decisions, however, would likely result in successive mis-predictions over diverse images that may exist in real-life scenarios.