paperbot · PL 论文追踪

全部论文 待读 收藏

P2 AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests

AssertFlip:通过反转 LLM 生成的通过测试来复现 Bug
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 3/10 · 论文属于LLM辅助测试生成与缺陷复现,偏向动态测试与调试,而非用户核心关注的静态分析、形式化验证、浮点程序分析或贝叶斯程序分析;仅与“LLM作为工具改进程序分析”有较弱的泛化关联。引用 16 inbox
Lara Khatib, Noble Saji Mathews, Meiyappan Nagappan
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:AssertFlip 先生成通过测试再反转为失败测试,用 LLM 自动复现 Bug,在 SWT-Bench 上达到最佳效果。

问题:开源与工业环境中多数 bug 在报告时缺少可执行复现测试,使诊断和修复困难且耗时。方法:AssertFlip 利用 LLM 自动生成 Bug Reproducible Tests(BRT)。与直接生成失败测试不同,它先在含 bug 的行为上生成可通过的测试,再反转这些测试,使其在 bug 存在时失败;其假设是 LLM 更擅长编写通过测试。评估:在专为 BRT 构建的 SWT-Bench 上,AssertFlip 在 SWT-Bench-Verified 子集取得 43.6% 的 fail-to-pass 成功率,优于所有已知技术并位居 leaderboard 最佳。

问题
许多 bug 在报告时没有可执行测试来复现,导致诊断与修复更加困难且耗时。
方法
AssertFlip 使用 LLM 首先生成在含 bug 行为上通过的测试,再反转这些测试使其在 bug 存在时失败,从而得到 Bug Reproducible Tests。
评估
在 SWT-Bench 基准及其 SWT-Bench-Verified 子集上评估,AssertFlip 取得 43.6% 的 fail-to-pass 成功率,并优于所有已知技术。
原文摘要(Abstract)

Bug reproduction is critical in the software debugging and repair process, yet the majority of bugs in open-source and industrial settings lack executable tests to reproduce them at the time they are reported, making diagnosis and resolution more difficult and time-consuming. To address this challenge, we introduce AssertFlip, a novel technique for automatically generating Bug Reproducible Tests (BRTs) using large language models (LLMs). Unlike existing methods that attempt direct generation of failing tests, AssertFlip first generates passing tests on the buggy behaviour and then inverts these tests to fail when the bug is present. We hypothesize that LLMs are better at writing passing tests than ones that crash or fail on purpose. Our results show that AssertFlip outperforms all known techniques in the leaderboard of SWT-Bench, a benchmark curated for BRTs. Specifically, AssertFlip achieves a fail-to-pass success rate of 43.6% on the SWT-Bench-Verified subset.

链接与引用

DOI 原文 · arXiv · DBLP

BibTeX
@inproceedings{abs-2507-17542,
  title = {AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests},
  author = {Lara Khatib and Noble Saji Mathews and Meiyappan Nagappan},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3773127}
}