一句话：提出基于蜕变测试的方法，为SVM和深度学习图像分类器构建蜕变关系，识别实现缺陷。
机器学习在计算机视觉等任务上表现突出，但测试此类应用极具挑战且成本高昂。本文首先梳理测试ML应用的难点，继而提出基于蜕变测试的解决方案，针对基于支持向量机和深度学习的图像分类器分别设计蜕变关系，以在无预期输出的情况下检测实现缺陷。实证验证表明，该方法能够捕获ML应用中71%的实现缺陷，说明蜕变测试在ML应用测试中具有实用价值。
We have recently witnessed tremendous success of Machine Learning (ML) in practical applications. Computer vision, speech recognition and language translation have all seen a near human level performance. We expect, in the near future, most business applications will have some form of ML. However, testing such applications is extremely challenging and would be very expensive if we follow today's methodologies. In this work, we present an articulation of the challenges in testing ML based applications. We then present our solution approach, based on the concept of Metamorphic Testing, which aims to identify implementation bugs in ML based image classifiers. We have developed metamorphic relations for an application based on Support Vector Machine and a Deep Learning based application. Empirical validation showed that our approach was able to catch 71% of the implementation bugs in the ML applications.
@inproceedings{paperbot4814,
title = {Identifying implementation bugs in machine learning based image classifiers using metamorphic testing},
author = {Anurag Dwarakanath and Manish Ahuja and Samarth Sikand and Raghotham M. Rao and R. P. Jagadeesh Chandra Bose and Neville Dubash and Sanjay Podder},
booktitle = {ISSTA},
year = {2018},
doi = {10.1145/3213846.3213858}
}