paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Unveiling Hidden DNN Defects with Decision-Based Metamorphic Testing

基于决策的蜕变测试揭示 DNN 隐藏缺陷
ASE Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering2022匹配度 3/10 · 论文研究 DNN 蜕变测试与隐藏缺陷检测,属于机器学习系统的测试与调试,与用户关注的静态分析、Agentic/LLM 程序分析、浮点程序分析及贝叶斯程序分析等核心方向仅有边缘关联。引用 19 inbox
Yuanyuan Yuan, Qi Pang, Shuai Wang
crossref: 2026-09-12

AI 速览

一句话:提出基于决策的蜕变测试,暴露 DNN 依赖错误决策却偶然保持输出一致的隐藏缺陷。

现有 DNN 测试常采用蜕变测试(MT),通过语义保持变异输入图像并检查预测一致性。然而,DNN 可能依赖错误的决策(即输入上的某些组件)进行预测,却因偶然因素仍保持输出不变;这类缺陷会被现有 MT 框架忽略。更严重的是,错误决策可能在真实场景中多样图像上导致连续误预测。本文提出基于决策的蜕变测试,不再仅检查输出一致性,而是针对 DNN 决策所依赖的组件施加语义保持变异,以暴露隐藏缺陷。摘要未给出具体实验细节,但该工作指出现有框架的盲区,并论证基于决策的测试对发现此类 DNN 缺陷的必要性。

问题
现有蜕变测试主要关注语义保持变异前后输出预测是否一致,因而可能漏掉 DNN 依赖错误决策区域却偶然保持输出正确的隐藏缺陷。
方法
提出基于决策的蜕变测试,分析 DNN 预测所依赖的输入组件,并对这些组件施加语义保持变异,以检测导致错误决策的 DNN 缺陷。
评估
摘要未详述具体实验设计;论文通过揭示现有 MT 框架会忽略“错误决策但输出偶然一致”的缺陷,论证基于决策的测试对暴露此类 DNN 缺陷的必要性。
原文摘要(Abstract)

Contemporary DNN testing works are frequently conducted using metamorphic testing (MT). In general, de facto MT frameworks mutate DNN input images using semantics-preserving mutations and determine if DNNs can yield consistent predictions. Nevertheless, we find that DNNs may rely on erroneous decisions (certain components on the DNN inputs) to make predictions, which may still retain the outputs by chance. Such DNN defects would be neglected by existing MT frameworks. Erroneous decisions, however, would likely result in successive mis-predictions over diverse images that may exist in real-life scenarios.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot7311,
  title = {Unveiling Hidden DNN Defects with Decision-Based Metamorphic Testing},
  author = {Yuanyuan Yuan and Qi Pang and Shuai Wang},
  booktitle = {ASE},
  year = {2022},
  doi = {10.1145/3551349.3561157}
}