一句话：提出BLAST，结合LLM与搜索式测试从issue-patch对自动生成问题复现测试，优于SOTA并经真实仓库部署验证。
软件开发中，开发者常提交补丁却不附带可复现问题的测试，因此自动生成问题复现测试具有重要价值。本文提出BLAST，将LLM与基于搜索的软件测试（SBST）结合：LLM部分利用Git历史、静态分析和SBST生成测试来补充issue描述与补丁的上下文；SBST部分则把issue描述和补丁经由LLM生成的中间种子反序列化为SBST可用形式，驱动搜索优化。在精心构建的Python基准上，BLAST为151/426（35.4%）的issue成功生成问题复现测试，优于当前最优方法（23.5%）。作者还构建GitHub bot，在三个开源仓库部署三个月，处理32个PR-issue对，其中11例生成测试并提交给开发者，并通过反馈讨论研究挑战与机会。
Issue-reproducing tests fail on buggy code and pass once a patch is applied, thus increasing developers’ confidence that the issue has been resolved and will not be re-introduced. However, past research has shown that developers often commit patches without such tests, making the automated generation of issue-reproducing tests an area of interest. We propose BLAST, a tool for automatically generating issue-reproducing tests from issue-patch pairs by combining LLMs and search-based software testing (SBST). For the LLM part, we complement the issue description and the patch by extracting relevant context through Git history analysis, static analysis, and SBST-generated tests. For the SBST part, we adapt SBST for generating issue-reproducing tests; the issue description and the patch are fed into the SBST optimization through an intermediate LLM-generated seed, which we deserialize into SBST-compatible form. BLAST successfully generates issue-reproducing tests for 151/426 (35.4%) of the issues from a curated Python benchmark, outperforming the state-of-the-art (23.5%).Additionally, to measure the real-world impact of BLAST, we built a GitHub bot that runs BLAST whenever a new pull request (PR) linked to an issue is opened, and if BLAST generates an issue-reproducing test, the bot proposes it as a comment in the PR. We deployed the bot in three open-source repositories for three months, gathering data from 32 PRs-issue pairs. BLAST generated an issue-reproducing test in 11 of these cases, which we proposed to the developers. By analyzing the developers’ feedback, we discuss challenges and opportunities for researchers and tool builders.Data and material: https://doi.org/10.5281/zenodo.16949042
@inproceedings{paperbot9483,
title = {Automated Generation of Issue-Reproducing Tests by Combining LLMs and Search-Based Testing},
author = {Konstantinos Kitsios and Marco Castelluccio and Alberto Bacchelli},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00165}
}