一句话：提出LLM驱动的静态方法，从API规范中挖掘响应体约束作为oracle并生成RESTful API测试用例，以观察-确认提示抑制幻觉。
现有RESTful API测试中，响应体逻辑约束（作为oracle）的提取多依赖动态执行被测API，适用范围受限。本文提出RBCTest，一种互补的、基于LLM的静态方法：让大语言模型理解API规范，从中挖掘响应体应满足的约束，并据此自动生成测试用例。为缓解LLM的幻觉问题，作者设计了Observation-Confirmation（OC）提示方案，先用初始提示对约束进行语境化，再由后续提示更准确地确认约束是否存在。实验表明，采用OC提示的RBCTest在约束挖掘上平均精度达85.1%–93.6%，由约束生成的测试用例精度达86.4%–91.7%。作者还将其用于19个真实API，检测出46处API规范与实际响应数据之间的不一致，其中4处已在开发者论坛中被报告。
In API testing, deriving logical constraints on API response bodies to be used as oracles is crucial in generating test cases and performing automated testing of RESTful APIs. However, existing approaches are restricted to dynamic analysis in which oracles are extracted via the execution of APIs as part of the system under test. In this paper, we propose a complementary LLM-based, static approach in which the constraints for API response bodies are mined from API specifications. We leverage large language models (LLMs) to comprehend the API specifications, mine constraints for response bodies, and generate test cases. To reduce LLMs’ hallucination, we apply an Observation-Confirmation (OC) scheme which uses initial prompts to contextualize constraints, allowing subsequent prompts to more accurately confirm their presence. Our empirical results show that RBCTest with OC prompting achieves high precision in constraint mining with the average from 85.1%–93.6%. It also performs well in generating test cases from mined constraints, with a precision from 86.4%–91.7%. We also use the test cases generated by RBCTest to detect 46 mismatches between the API specification and actual response data for 19 real-world APIs. Four of the mismatches were, in fact, reported in developers’ forums.
@inproceedings{paperbot11949,
title = {RBCTest: Leveraging LLMs to Mine and Verify Oracles of API Response Bodies for RESTful API Testing},
author = {Hieu Huynh and Quoc-Tri Le and Tu Nguyen and Viet Nguyen and Vu Nguyen and Tien N. Nguyen},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3787775}
}