一句话:提出 DRFuzz,用多样性导向测试准则与 GAN 生成高保真输入,以发现深度学习系统版本间的回归故障。
深度学习系统在版本演化过程中与传统软件一样会引入回归故障,但现有方法难以在发现尽可能多样的错误行为的同时保证生成输入的保真度。本文提出回归模糊测试技术 DRFuzz:一方面设计多样性导向的测试准则,引导测试尽可能探索不同的故障行为以提升所发现回归故障的多样性;另一方面引入 GAN 模型对生成过程加以约束,保证生成测试输入的真实性与保真度。作者在 4 个深度学习系统对象的 4 种回归场景下开展了大规模实验,并与两个当前最优方法进行对比。结果表明,DRFuzz 在检测到的回归故障数量上平均分别提升 1177% 与 539%。
Deep learning (DL) Systems have been widely used in various domains. Similar to traditional software, DL system evolution may also incur regression faults. To find the regression faults between versions of a DL system, we propose a novel regression fuzzing technique called DRFuzz, which facilitates generating inputs that trigger diverse regression faults and have high fidelity. To enhance the diversity of the found regression faults, DRFuzz proposes a diversity-oriented test criterion to explore as many faulty behaviors as possible. Then, DRFuzz incorporates the GAN model to guarantee the fidelity of generated test inputs. We conduct an extensive study on four subjects in four regression scenarios of DL systems. The experimental results demonstrate the superiority of DRFuzz over the two compared state-of-the-art approaches, with an average improvement of 1,177% and 539% in terms of the number of detected regression faults.