一句话:提出DFV,通过学习输入分布模型并前置到DNN,将证伪与验证聚焦于相关输入空间,避免不可行输入。
现有DNN验证与证伪方法不考虑输入分布,浪费算力于无关输入并报告虚假的属性违规。本文借鉴传统系统的基于模型验证思想,提出DFV:利用无监督学习自动构建输入分布模型,将其前缀到DNN上,迫使所有输入来自学习到的分布,并将属性重述到分布模型的输入空间。这一转换使现有DNN证伪与验证工具能够针对输入分布,避免考虑不可行输入。实验使用7种证伪与验证工具、两个不同数据集上的DNN以及93个不同的分布模型,结果表明工具找到的反例更能代表数据分布,且DFV的性能随领域、模型和工具而变化。
DNN validation and verification approaches that are input distribution agnostic waste effort on irrelevant inputs and report false property violations. Drawing on the large body of work on model-based validation and verification of traditional systems, we introduce the first approach that leverages environmental models to focus DNN falsification and verification on the relevant input space. Our approach, DFV, automatically builds an input distribution model using unsupervised learning, prefixes that model to the DNN to force all inputs to come from the learned distribution, and reformulates the property to the input space of the distribution model. This transformed verification problem allows existing DNN falsification and verification tools to target the input distribution – avoiding consideration of infeasible inputs. Our study of DFV with 7 falsification and verification tools, two DNNs defined over different data sets, and 93 distinct distribution models, provides clear evidence that the counterexamples found by the tools are much more representative of the data distribution, and it shows how the performance of DFV varies across domains, models, and tools.