一句话：提出并落地于字节跳动的 LLM 驱动微服务 API 全自动测试系统，自动生成测试请求并检测崩溃与功能缺陷。
微服务架构下 API 数量庞大，逐一人工测试代价极高，亟需自动化测试手段以保证每个 API 的功能正确与可靠。本文提出 TestGPT-Server，一个在字节跳动开发并部署的全自动微服务 API 测试系统，利用大语言模型增强测试请求生成与缺陷检测。系统首先用 LLM 增强的接口分析器和代码分析器抽取被测 API 的语法与语义信息，再据此以定向请求生成器和引导式随机 fuzzer 生成覆盖更全面的测试请求；请求发送后，由 LLM 辅助的响应分析器识别潜在的服务崩溃，并用断言挖掘器自动挖掘功能断言以发现功能缺陷。在字节跳动的部署评估表明，该系统相比现有测试请求有 40.52% 的提升，在 14,366 个微服务 API 上发现 50 个崩溃缺陷与 2 个功能缺陷，且人工投入极少。
Despite the advantages of the microservice architecture, there is an urgent need for automated testing microservice APIs, ensuring that each API functions correctly and reliably without exhaustive manual effort. This paper introduces TestGPT-Server, a novel, fully automated testing system for microservice APIs developed and deployed at ByteDance, leveraging the capabilities of large language models (LLMs) to enhance test request generation and bug detection. TestGPT-Server utilizes an LLM-enhanced interface analyzer and a code analyzer to extract syntactic and semantic information of the API under test. Based on the extracted information, it employs a targeted request generator and a guided stochastic fuzzer to produce comprehensive test requests. After sending generated test requests to the API, an LLM-assisted response analyzer identifies potential service crashes, and an assertion miner automatically mines functional assertions to detect functional bugs. Deployed at ByteDance, the evaluation results demonstrate that TestGPT-Server is effective, achieving a 40.52% improvement over existing test requests and identifying 50 crash bugs and 2 functional bugs across 14,366 microservice APIs, with minimal manual effort.
DOI 原文 ·
@inproceedings{paperbot9012,
title = {TestGPT-Server: Automatically Testing Microservices with Large Language Models at ByteDance},
author = {Jue Wang and Shuxiang Chen and Yu Liu and Yuan Deng and Lei Zhang and Yuanchang Fu and Bo Liu},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3728545}
}