paperbot · PL 论文追踪

全部论文 待读 收藏

P0 HarnessLLM: Rust Verification Harness Generation with Large Language Models

HarnessLLM:用大语言模型生成 Rust 验证 harness
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 8/10 · 该论文属于“AI→PL”,用LLM自动生成Rust形式化验证所需的harness,直接命中“利用LLM/Agent改进静态分析或形式化验证”的核心方向;其问题驱动、面向真实Rust项目并报告实际内存安全缺陷检出,兼具工程落地价值。引用 0 inbox
Minghua Wang, Yuwei Liu, Lin Huang
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:提出 HarnessLLM,从既有测试套件自动抽取调用场景并用 LLM 增量生成、迭代修复 Rust 验证 harness,检出 6 个真实内存安全缺陷。

Rust 的所有权与类型系统提供了强内存安全保障,但 unsafe 代码与运行时 panic 仍构成显著风险,而形式化验证所需的验证 harness 往往依赖人工编写。直接将 LLM 用于 harness 生成会出现 API 调用不准确、非确定性数据生成低效以及编造修复等问题。本文提出 HarnessLLM:它从现有测试用例中自动抽取调用场景,基于依赖分析生成非确定性参数,增量式合成 harness,并通过迭代精化保留关键代码区域、将编造的类型或函数反馈给 LLM 以纠正。在 9 个真实 Rust 代码库上,该方法从 494 个测试用例中抽取 294 个调用场景,精度达 94.66%,平均每个场景 145 秒生成 harness,显著优于仅能处理 41% 场景的 Autoharness;最终利用生成的 harness 检出 6 个真实内存安全缺陷。据作者所知,这是首个用 LLM 为真实 Rust 项目生成内存安全验证 harness 的工作。

问题
Rust 中 unsafe 代码与运行时 panic 仍存在内存安全风险,形式化验证所需的 harness 编写困难且高度依赖人工;直接让 LLM 生成 harness 会出现 API 调用不准确、非确定性数据生成低效与编造修复等问题。
方法
HarnessLLM 从既有测试用例中抽取调用场景,基于依赖分析生成非确定性参数,增量合成 harness,并迭代精化:保留关键代码区域,同时把编造的类型或函数报告给 LLM 进行纠正。
评估
在 9 个真实 Rust 代码库、494 个测试用例上评估:抽取 294 个调用场景,精度 94.66%,平均每场景 145 秒生成 harness,优于仅覆盖 41% 场景的 Autoharness;生成的 harness 最终检出 6 个真实内存安全缺陷。
原文摘要(Abstract)

Rust’s ownership and type system offer strong memory safety guarantees, but unsafe code and runtime panics still present significant risks. Formal verification is essential to ensure memory safety, but developing verification harnesses remains a challenging and manual task. Although large language models (LLMs) have shown strong performance in various code analysis tasks, directly applying them to harness generation often results in inaccurate API invocations, inefficient nondeterministic data generation, and fabricated fixes. In this paper, we present HarnessLLM, an automated workflow that leverages LLMs to generate verification harnesses for Rust code directly from existing test suites. HarnessLLM automatically extracts calling scenarios from test cases, generates nondeterministic arguments based on dependency analysis, and incrementally synthesizes harnesses. It then iteratively refines the harnesses, preserving critical code regions and reporting fabricated types or functions to LLMs for correction. In our evaluation on 9 real-world Rust codebases, HarnessLLM extracted 294 calling scenarios from 494 test cases with 94.66% precision and generated harnesses for all scenarios in an average of 145 seconds each. It outperformed the existing approach, Autoharness, which succeeded on only 41% of those scenarios. Finally, 6 real-world memory safety bugs were detected using the generated harnesses, demonstrating the practical utility of our approach in verification. To our knowledge, this is the first work to use LLMs for generating harnesses aimed at memory safety verification in real-world Rust projects.

链接与引用

DOI 原文 · arXiv · DBLP

BibTeX
@inproceedings{abs-2607-22161,
  title = {HarnessLLM: Rust Verification Harness Generation with Large Language Models},
  author = {Minghua Wang and Yuwei Liu and Lin Huang},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3773122}
}