一句话：提出iAudit，将两阶段微调与LLM智能体结合，用于带理由的智能合约漏洞审计。
智能合约是区块链上的去中心化应用。现有LLM审计智能合约的效果有限，即便GPT-4在决策与理由均正确时精度也仅约30%，主因是通用模型未针对Solidity审计领域微调。本文提出iAudit，结合微调与LLM智能体：先两阶段微调Detector做判断、Reasoner生成漏洞原因，再引入Ranker和Critic两个智能体迭代选择与辩论最优原因。作者用1,734个正样本和1,810个负样本微调，并与CodeBERT等微调模型及GPT-4等提示模型比较。在263个真实漏洞上，iAudit达到91.21% F1和91.11%准确率，生成原因与真实原因一致性约38%。
Smart contracts are decentralized applications built atop blockchains like Ethereum. Recent research has shown that large language models (LLMs) have potential in auditing smart contracts, but the state-of-the-art indicates that even GPT-4 can achieve only 30% precision (when both decision and justification are correct). This is likely because off-the-shelf LLMs were primarily pre-trained on a general text/code corpus and not fine-tuned on the specific domain of Solidity smart contract auditing. In this paper, we propose iAudit, a general framework that combines fine-tuning and LLM-based agents for intuitive smart contract auditing with justifications. Specifically, iAudit is inspired by the observation that expert human auditors first perceive what could be wrong and then perform a detailed analysis of the code to identify the cause. As such, iAudit employs a two-stage fine-tuning approach: it first tunes a Detector model to make decisions and then tunes a Reasoner model to generate causes of vulnerabilities. However, fine-tuning alone faces challenges in accurately identifying the optimal cause of a vulnerability. Therefore, we introduce two LLM-based agents, the Ranker and Critic, to iteratively select and debate the most suitable cause of vulnerability based on the output of the fine-tuned Reasoner model. To evaluate iAudit, we collected a balanced dataset with 1,734 positive and 1,810 negative samples to fine-tune iAudit. We then compared it with traditional fine-tuned models (CodeBERT, GraphCodeBERT, CodeT5, and UnixCoder) as well as prompt learning-based LLMs (GPT4, GPT-3.5, and CodeLlama-13b/34b). On a dataset of 263 real smart contract vulnerabilities, iAudit achieves an F1 score of 91.21% and an accuracy of 91.11%. The causes generated by iAudit achieved a consistency of about 38% compared to the ground truth causes.
DOI 原文 ·
@inproceedings{paperbot8906,
title = {Combining Fine-Tuning and LLM-Based Agents for Intuitive Smart Contract Auditing with Justifications},
author = {Wei Ma and Daoyuan Wu and Yuqiang Sun and Tianwen Wang and Shangqing Liu and Jian Zhang and Yue Xue and Yang Liu},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00027}
}