一句话:提出 Morest,用动态更新的 RESTful 服务属性图建模 API 行为并引导调用序列生成,显著提升黑盒测试的覆盖率与缺陷检出能力。
RESTful API 黑盒测试的难点在于构造正确的调用序列以做深度测试。已有基于 OpenAPI 规范学习 API 依赖的方法,往往缺乏对全部 API 连接关系的整体认知,也难以自适应修正已学知识。本文提出 Morest,一种基于模型的 RESTful API 测试技术:构建并维护动态更新的 RESTful 服务属性图(RPG),用以刻画服务行为并指导调用序列生成。评估显示,相比当前最优技术,Morest 平均多请求 152.66%–232.45% 的 API 操作、多覆盖 26.16%–103.24% 的代码行、多检出 40.64%–215.94% 的缺陷;在 6 个真实项目中发现 44 个缺陷(13 个已有方法无法检出),其中 2 个来自 Bitbucket。
RESTful APIs are arguably the most popular endpoints for accessing Web services. Blackbox testing is one of the emerging techniques for ensuring the reliability of RESTful APIs. The major challenge in testing RESTful APIs is the need for correct sequences of API operation calls for in-depth testing. To build meaningful operation call sequences, researchers have proposed techniques to learn and utilize the API dependencies based on OpenAPI specifications. However, these techniques either lack the overall awareness of how all the APIs are connected or the flexibility of adaptively fixing the learned knowledge. In this paper, we propose Morest, a model-based RESTful API testing technique that builds and maintains a dynamically updating RESTful-service Property Graph (RPG) to model the behaviors of RESTful-services and guide the call sequence generation. We empirically evaluated Morest and the results demonstrate that Morest can successfully request an average of 152.66%-232.45% more API operations, cover 26.16%-103.24% more lines of code, and detect 40.64%-215.94% more bugs than state-of-the-art techniques. In total, we applied Morest to 6 real-world projects and found 44 bugs (13 of them cannot be detected by existing approaches). Specifically, 2 of the confirmed bugs are from Bitbucket, a famous code management service with more than 6 million users.