paperbot · PL 论文追踪

全部论文 待读 收藏

P1 SAINT: Service-level Integration Test Generation with Program Analysis and LLM-based Agents

SAINT:结合程序分析与LLM智能体的服务级集成测试生成
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 6/10 · 论文结合静态分析与LLM智能体生成企业Java服务级测试,属于程序分析与LLM的交叉方向,且有工业案例和问题驱动特点;但其核心是测试生成,而非用户重点关注的Agentic程序静态分析、LLM改进静态分析/形式化验证,也不涉及浮点或贝叶斯程序分析,因此匹配度中等。引用 2 inbox
Rangeet Pan, Raju Pavuluri, Ruikai Huang, Tyler Stennett, Rahul Krishna, Alessandro Orso, Saurabh Sinha
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:提出SAINT,用静态分析构建端点模型与操作依赖图,并借助LLM智能体自动生成企业Java服务级测试。

企业应用需服务级测试,但现有RESTful API工具多依赖模糊测试或OpenAPI规范,真实代码库常缺少规范,且难以生成有意义的功能场景测试。SAINT是面向企业Java应用的白盒方法,结合静态分析、LLM与智能体,构建端点模型和操作依赖图,刻画端点语义与跨端点顺序约束。智能体据此生成端点级测试以最大化代码和数据库交互覆盖,并从代码提取用例,经规划、行动、反思循环精化为场景测试。在八个Java应用(含专有企业应用)上评估,SAINT在覆盖率、故障检测与场景生成上有效,开发者调研认可其场景测试。

问题
企业级服务测试常缺乏OpenAPI规范,现有工具依赖模糊测试,难以生成能有效覆盖有意义功能场景的测试。
方法
SAINT采用白盒方法,结合静态分析、LLM和LLM智能体,构建端点模型与操作依赖图,并生成端点级和场景级测试。
评估
在八个Java应用(包括一个专有企业应用)上评估覆盖率、故障检测和场景生成效果,并通过开发者调研验证场景测试的可用性。
原文摘要(Abstract)

Enterprise applications are typically tested at multiple levels, with service-level testing playing an important role in validating application functionality. Existing service-level testing tools, especially for RESTful APIs, often employ fuzzing and/or depend on OpenAPI specifications which are not readily available in real-world enterprise codebases. Moreover, these tools are limited in their ability to generate functional tests that effectively exercise meaningful scenarios. In this work, we present saint, a novel white-box testing approach for service-level testing of enterprise Java applications. saint combines static analysis, large language models (LLMs), and LLM-based agents to automatically generate endpoint and scenario-based tests. The approach builds two key models: an endpoint model, capturing syntactic and semantic information about service endpoints, and an operation dependency graph, capturing inter-endpoint ordering constraints. saint then employs LLM-based agents to generate tests. Endpoint-focused tests aim to maximize code and database interaction coverage. Scenario-based tests are synthesized by extracting application use cases from code and refining them into executable tests via planning, action, and reflection phases of the agentic loop. We evaluated saint on eight Java applications, including a proprietary enterprise application. Our results illustrate the effectiveness of saint in coverage, fault detection, and scenario generation. Moreover, a developer survey provides strong endorsement of the scenario-based tests generated by saint. Overall, our work shows that combining static analysis with agentic LLM workflows enables more effective, functional, and developer-aligned service-level test generation.

链接与引用

DOI 原文 · arXiv · DBLP

BibTeX
@inproceedings{abs-2511-13305,
  title = {SAINT: Service-level Integration Test Generation with Program Analysis and LLM-based Agents},
  author = {Rangeet Pan and Raju Pavuluri and Ruikai Huang and Tyler Stennett and Rahul Krishna and Alessandro Orso and Saurabh Sinha},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3773185}
}