一句话：用深度强化学习从黑盒交互中挖掘 REST API 的隐式约束，自动生成高覆盖、易检错的测试用例。
REST API 的自动化测试对构建可信的 Web 系统至关重要，但现有黑盒方法严重依赖 API 的形式化文档（Open API Specification）。OAS 主要刻画语法层面的信息，如操作间的生产者—消费者关系、输入值的属性以及自然语言描述的附加约束，却无法揭示 API 的业务逻辑语义，例如操作之间隐含的时序（逻辑依赖）关系与隐式的输入值约束，这使黑盒工具难以自动生成真正有效的测试用例。本文提出 DeepREST，一种面向 REST API 的黑盒自动测试方法，利用深度强化学习去发现文档中未显式给出的隐式 API 约束。方法以好奇心驱动的学习引导智能体探索 API，习得有效的操作测试顺序，从而先把 API 带入可测试状态并避免后续交互失败；同时把成功交互中积累的经验用于精确的输入数据生成，决定使用哪些参数以及如何取值；此外，DeepREST 在探索与利用之间交替，通过变异已成功的 API 交互来提升测试覆盖率并收集更多经验。实证验证表明，该方法在测试覆盖率与故障检测方面效果显著，并优于当前最先进的基线方法。
Automatically crafting test scenarios for REST APIs helps deliver more reliable and trustworthy web-oriented systems. However, current black-box testing approaches rely heavily on the information available in the API’s formal documentation, i.e., the Open API Specification (OAS for short). While useful, the OAS mostly covers syntactic aspects of the API (e.g., producer-consumer relations between operations, input value properties, and additional constraints in natural language), and it lacks a deeper understanding of the API business logic. Missing semantics include implicit ordering (logic dependency) between operations and implicit input-value constraints. These limitations hinder the ability of black-box testing tools to generate truly effective test cases automatically.This paper introduces DeepREST, a novel black-box approach for automatically testing REST APIs. It leverages deep reinforcement learning to uncover implicit API constraints, that is, constraints hidden from API documentation. Curiosity-driven learning guides an agent in the exploration of the API and learns an effective order to test its operations. This helps identify which operations to test first to take the API in a testable state and avoid failing API interactions later. At the same time, experience gained on successful API interactions is leveraged to drive accurate input data generation (i.e., what parameters to use and how to pick their values). Additionally, DeepREST alternates exploration with exploitation by mutating successful API interactions to improve test coverage and collect further experience.Our empirical validation suggests that the proposed approach is very effective in achieving high test coverage and fault detection and superior to a state-of-the-art baseline.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{CorradiniMPC24,
title = {DeepREST: Automated Test Case Generation for REST APIs Exploiting Deep Reinforcement Learning},
author = {Davide Corradini and Zeno Montolli and Michele Pasqua and Mariano Ceccato},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695511}
}