一句话：针对依赖 ML API 的软件，提出自动化测试需自动生成相关输入、判定输出并区分错误来源。
越来越多软件应用使用机器学习（ML）方案完成认知任务，这些 ML API 在统计上模仿人类行为。测试此类软件面临两难：需要大量人工设计图像、文本或音频输入，使其与软件场景相关；还需要人工判断软件是否像多数人一样处理这些输入。即便发现异常行为，也往往难以判断问题出在认知 ML API 内部，还是出在使用该 API 的代码中。该论文标题表明其关注使用 ML API 的软件的自动化测试，但所给摘要主要陈述上述挑战，未提供具体方法、工具或评估结果。
An increasing number of software applications incorporate machine learning (ML) solutions for cognitive tasks that statistically mimic human behaviors. To test such software, tremendous human effort is needed to design image/text/audio inputs that are relevant to the software, and to judge whether the software is processing these inputs as most human beings do. Even when misbehavior is exposed, it is often unclear whether the culprit is inside the cognitive ML API or the code using the API.
@inproceedings{WanLXLHM022,
title = {Automated testing of software that uses machine learning APIs},
author = {Chengcheng Wan and Shicheng Liu and Sophie Xie and Yifan Liu and Henry Hoffmann and Michael Maire and Shan Lu},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510068}
}