一句话：提出覆盖率引导的 DNN 模糊测试框架 DeepHunter，通过变形变异和种子选择检测 DNN 缺陷。
深度神经网络在自动驾驶等安全攸关场景中可能出现由隐藏缺陷导致的错误行为。论文提出 DeepHunter，一个面向通用 DNN 的覆盖率引导模糊测试框架。它设计变形变异策略生成语义保持的新测试，并用多种可扩展覆盖准则作为反馈指导测试生成；同时结合基于多样性和新近性的种子选择。作者实现并集成5种测试准则和4种种子选择策略，开展大规模实验。结果表明，变形变异生成有效测试的比率最高达98%；多样性种子选择通常比新近性更重要；DeepHunter在覆盖率和发现缺陷的数量与多样性上优于现有方法；在角区域准则引导下可捕获DNN量化迁移中的缺陷。
The past decade has seen the great potential of applying deep neural network (DNN) based software to safety-critical scenarios, such as autonomous driving. Similar to traditional software, DNNs could exhibit incorrect behaviors, caused by hidden defects, leading to severe accidents and losses. In this paper, we propose DeepHunter, a coverage-guided fuzz testing framework for detecting potential defects of general-purpose DNNs. To this end, we first propose a metamorphic mutation strategy to generate new semantically preserved tests, and leverage multiple extensible coverage criteria as feedback to guide the test generation. We further propose a seed selection strategy that combines both diversity-based and recency-based seed selection. We implement and incorporate 5 existing testing criteria and 4 seed selection strategies in DeepHunter. Large-scale experiments demonstrate that (1) our metamorphic mutation strategy is useful to generate new valid tests with the same semantics as the original seed, by up to a 98% validity ratio; (2) the diversity-based seed selection generally weighs more than recency-based seed selection in boosting the coverage and in detecting defects; (3) DeepHunter outperforms the state of the arts by coverage as well as the quantity and diversity of defects identified; (4) guided by corner-region based criteria, DeepHunter is useful to capture defects during the DNN quantization for platform migration.
@inproceedings{paperbot5346,
title = {DeepHunter: a coverage-guided fuzz testing framework for deep neural networks},
author = {Xiaofei Xie and Lei Ma and Felix Juefei-Xu and Minhui Xue and Hongxu Chen and Yang Liu and Jianjun Zhao and Bo Li and Jianxiong Yin and Simon See},
booktitle = {ISSTA},
year = {2019},
doi = {10.1145/3293882.3330579}
}