一句话:提出 HarnessLLM,从既有测试套件自动抽取调用场景并用 LLM 增量生成、迭代修复 Rust 验证 harness,检出 6 个真实内存安全缺陷。
Rust 的所有权与类型系统提供了强内存安全保障,但 unsafe 代码与运行时 panic 仍构成显著风险,而形式化验证所需的验证 harness 往往依赖人工编写。直接将 LLM 用于 harness 生成会出现 API 调用不准确、非确定性数据生成低效以及编造修复等问题。本文提出 HarnessLLM:它从现有测试用例中自动抽取调用场景,基于依赖分析生成非确定性参数,增量式合成 harness,并通过迭代精化保留关键代码区域、将编造的类型或函数反馈给 LLM 以纠正。在 9 个真实 Rust 代码库上,该方法从 494 个测试用例中抽取 294 个调用场景,精度达 94.66%,平均每个场景 145 秒生成 harness,显著优于仅能处理 41% 场景的 Autoharness;最终利用生成的 harness 检出 6 个真实内存安全缺陷。据作者所知,这是首个用 LLM 为真实 Rust 项目生成内存安全验证 harness 的工作。
Rust’s ownership and type system offer strong memory safety guarantees, but unsafe code and runtime panics still present significant risks. Formal verification is essential to ensure memory safety, but developing verification harnesses remains a challenging and manual task. Although large language models (LLMs) have shown strong performance in various code analysis tasks, directly applying them to harness generation often results in inaccurate API invocations, inefficient nondeterministic data generation, and fabricated fixes. In this paper, we present HarnessLLM, an automated workflow that leverages LLMs to generate verification harnesses for Rust code directly from existing test suites. HarnessLLM automatically extracts calling scenarios from test cases, generates nondeterministic arguments based on dependency analysis, and incrementally synthesizes harnesses. It then iteratively refines the harnesses, preserving critical code regions and reporting fabricated types or functions to LLMs for correction. In our evaluation on 9 real-world Rust codebases, HarnessLLM extracted 294 calling scenarios from 494 test cases with 94.66% precision and generated harnesses for all scenarios in an average of 145 seconds each. It outperformed the existing approach, Autoharness, which succeeded on only 41% of those scenarios. Finally, 6 real-world memory safety bugs were detected using the generated harnesses, demonstrating the practical utility of our approach in verification. To our knowledge, this is the first work to use LLMs for generating harnesses aimed at memory safety verification in real-world Rust projects.