一句话：提出基于 PAC 模型学习的黑盒框架，将模型学习融入 DNN 局部鲁棒性分析，提升实用性与可扩展性。
为分析深度神经网络（DNN）的局部鲁棒性，本文从模型学习视角提出实用框架。该方法采用带场景优化的黑盒模型学习，将 DNN 局部行为抽象为具有 PAC 保证的仿射模型，并推断 PAC 模型鲁棒性。其创新在于在模型层面而非样本分布上构造 PAC 保证，使鲁棒性评估更忠实准确。作者实现原型工具 DeepPAC，可扩展且高效，尤其适合复杂结构或高维输入。在 MNIST、CIFAR-10、ImageNet 的 20 个模型上，与形式验证、统计、测试和对抗攻击 4 类基线比较，DeepPAC 优于统计方法 PROVERO，比形式验证工具 ERAN 更实用，且结果与 DeepGini 等 DNN 测试工作一致。
To analyse local robustness properties of deep neural networks (DNNs), we present a practical framework from a model learning perspective. Based on black-box model learning with scenario optimisation, we abstract the local behaviour of a DNN via an affine model with the probably approximately correct (PAC) guarantee. From the learned model, we can infer the corresponding PAC-model robustness property. The innovation of our work is the integration of model learning into PAC robustness analysis: that is, we construct a PAC guarantee on the model level instead of sample distribution, which induces a more faithful and accurate robustness evaluation. This is in contrast to existing statistical methods without model learning. We implement our method in a prototypical tool named DeepPAC. As a black-box method, DeepPAC is scalable and efficient, especially when DNNs have complex structures or high-dimensional inputs. We extensively evaluate DeepPAC, with 4 baselines (using formal verification, statistical methods, testing and adversarial attack) and 20 DNN models across 3 datasets, including MNIST, CIFAR-10, and ImageNet. It is shown that DeepPAC outperforms the state-of-the-art statistical method PROVERO, and it achieves more practical robustness analysis than the formal verification tool ERAN. Also, its results are consistent with existing DNN testing work like DeepGini.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{LiYHS0Z22,
title = {Towards practical robustness analysis for DNNs based on PAC-model learning},
author = {Renjue Li and Pengfei Yang and Cheng-Chao Huang and Youcheng Sun and Bai Xue and Lijun Zhang},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510143}
}