一句话：提出 Fuzz4All，首个以 LLM 为输入生成与变异引擎、可跨多种输入语言通用的模糊测试器。
编译器、运行时、约束求解器等以编程或形式语言为输入的系统是软件开发的基础设施，但现有模糊测试器通常只针对某一特定语言，难以迁移到其他语言甚至同一语言的其他版本，且生成输入往往局限于已知语言特性，难以暴露新特性相关的缺陷。本文提出 Fuzz4All，其核心思想是把大语言模型（LLM）同时用作输入生成与变异引擎，从而为任意实际相关的语言产生多样且真实的测试输入。为此作者设计了自动提示（autoprompting）技术来构造适合模糊测试的 LLM 提示，并提出 LLM 驱动的模糊测试循环，通过迭代更新提示不断产生新的输入。作者在覆盖 C、C++、Go、SMT2、Java、Python 六种语言的九个被测系统上评估，结果显示通用模糊测试在全部六种语言上均取得比语言专用模糊测试器更高的覆盖率；同时 Fuzz4All 在 GCC、Clang、Z3、CVC5、OpenJDK 与 Qiskit 量子计算平台等广泛使用的系统中发现 98 个缺陷，其中 64 个已被开发者确认为此前未知的缺陷。
Fuzzing has achieved tremendous success in discovering bugs and vulnerabilities in various software systems. Systems under test (SUTs) that take in programming or formal language as inputs, e.g., compilers, runtime engines, constraint solvers, and software libraries with accessible APIs, are especially important as they are fundamental building blocks of software development. However, existing fuzzers for such systems often target a specific language, and thus cannot be easily applied to other languages or even other versions of the same language. Moreover, the inputs generated by existing fuzzers are often limited to specific features of the input language, and thus can hardly reveal bugs related to other or new features. This paper presents Fuzz4All, the first fuzzer that is universal in the sense that it can target many different input languages and many different features of these languages. The key idea behind Fuzz4All is to leverage large language models (LLMs) as an input generation and mutation engine, which enables the approach to produce diverse and realistic inputs for any practically relevant language. To realize this potential, we present a novel autoprompting technique, which creates LLM prompts that are well-suited for fuzzing, and a novel LLM-powered fuzzing loop, which iteratively updates the prompt to create new fuzzing inputs. We evaluate Fuzz4All on nine systems under test that take in six different languages (C, C++, Go, SMT2, Java, and Python) as inputs. The evaluation shows, across all six languages, that universal fuzzing achieves higher coverage than existing, language-specific fuzzers. Furthermore, Fuzz4All has identified 98 bugs in widely used systems, such as GCC, Clang, Z3, CVC5, OpenJDK, and the Qiskit quantum computing platform, with 64 bugs already confirmed by developers as previously unknown.
DOI 原文 ·
@inproceedings{paperbot11357,
title = {Fuzz4All: Universal Fuzzing with Large Language Models},
author = {Chunqiu Steven Xia and Matteo Paltenghi and Jia Le Tian and Michael Pradel and Lingming Zhang},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639121}
}