一句话：提出基于模型变异敏感度的运行时对抗样本检测方法，通过统计假设检验区分正常与对抗样本。
深度神经网络（DNN）在众多应用中表现优异，却易受对抗样本攻击：对正常样本施加人眼不可察觉的扰动即可导致错误预测，而现有防御机制大多效果有限。本文提出一种运行时对抗样本检测方法。核心观察是：对 DNN 施加随机变异后，对抗样本比正常样本更敏感。作者据此定义敏感度度量，并实证表明两类样本的敏感度具有可区分性；进而将统计假设检验与模型变异测试结合，在运行时通过测量输入样本的敏感度判断其更可能为正常还是对抗样本。在 MNIST 和 CIFAR10 数据集上的实验表明，该方法能高效、准确地检测由当前先进攻击方法生成的对抗样本。
Deep neural networks (DNN) have been shown to be useful in a wide range of applications. However, they are also known to be vulnerable to adversarial samples. By transforming a normal sample with some carefully crafted human imperceptible perturbations, even highly accurate DNN make wrong decisions. Multiple defense mechanisms have been proposed which aim to hinder the generation of such adversarial samples. However, a recent work show that most of them are ineffective. In this work, we propose an alternative approach to detect adversarial samples at runtime. Our main observation is that adversarial samples are much more sensitive than normal samples if we impose random mutations on the DNN. We thus first propose a measure of 'sensitivity' and show empirically that normal samples and adversarial samples have distinguishable sensitivity. We then integrate statistical hypothesis testing and model mutation testing to check whether an input sample is likely to be normal or adversarial at runtime by measuring its sensitivity. We evaluated our approach on the MNIST and CIFAR10 datasets. The results show that our approach detects adversarial samples generated by state-of-the-art attacking methods efficiently and accurately.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-1812-05793,
title = {Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing},
author = {Jingyi Wang and Guoliang Dong and Jun Sun and Xinyu Wang and Peixin Zhang},
booktitle = {ICSE},
year = {2019},
doi = {10.1109/icse.2019.00126}
}