一句话：提出 KAT，从 API 文档构建知识图谱导出约束并作为测试目标，引导搜索式测试自动生成断言以发现更多 API 缺陷。
搜索式软件测试（SBST）在分支覆盖、数据依赖覆盖等预设目标下能有效生成测试用例，但预定义目标难以适应多样化项目，无法高效检测更多故障。本文提出 KAT，一种知识感知的 SBST 方法，基于被测程序（PUT）所使用的 API 按需生成断言。KAT 从 API 文档构建 API 知识图谱，推导客户端代码需要满足的约束，并将每条约束作为程序分支插桩到 PUT 中，充当测试目标以引导 SBST 发现缺陷。作者在封闭世界与开放世界两种设置下，与 EvoSuite 和 Catcher 两个基线对比检测 API 相关缺陷：封闭世界实验中 KAT 的 F1 为 0.55，优于基线的 0.24 与 0.30；开放世界实验中，KAT 实际发现的缺陷数分别比基线多 59.64% 与 9.05%。
Search-based software testing (SBST) has proved its effectiveness in generating test cases to achieve its defined test goals, such as branch and data-dependency coverage. However, to detect more program faults in an effective way, pre-defined goals can hardly be adaptive in diversified projects. In this work, we propose KAT, a novel knowledge-aware SBST approach to generate on-demand assertions in the program under test (PUT) based on its used APIs. KAT constructs an API knowledge graph from the API documentation to derive the constraints that the client codes need to satisfy. Each constraint is instrumented into the PUT as a program branch, serving as a test goal to guide SBST to detect faults. We evaluate KAT with two baselines (i.e., EvoSuite and Catcher) with a close-world and an open-world experiment to detect API bugs. The close-world experiment shows that KAT outperforms the baselines in the F1-score (0.55 vs. 0.24 and 0.30) to detect API-related bugs. The open-world experiment shows that KAT can detect 59.64% and 9.05% more bugs than the baselines in practice.
@inproceedings{RenYLXLL23,
title = {API-Knowledge Aware Search-Based Software Testing: Where, What, and How},
author = {Xiaoxue Ren and Xinyuan Ye and Yun Lin and Zhenchang Xing and Shuqing Li and Michael R. Lyu},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3616269}
}