paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Green Fuzzer Benchmarking

绿色模糊测试器基准评测
ISSTA Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis2023匹配度 2/10 · 论文聚焦模糊测试器评测平台与加速,属于测试与基准评测方向;与用户核心关注的静态分析、LLM/Agent 与程序分析交叉、浮点程序分析和贝叶斯程序分析均无直接关联。引用 8 inbox
Jiradet Ounjai, Valentin Wüstholz, Maria Christakis
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:提出 GreenBench 平台,通过更多、更短的模糊测试基准,在几乎不损失排名准确性的前提下把评测速度提升 18–61 倍。

模糊测试在过去十年中因发现缺陷的高效性而广受关注,但模糊测试器的评测长期受制于基准测试缺乏标准化。2020 年 Google 发布的 FuzzBench 开源平台被广泛用于准确的模糊测试器基准评测,然而一次典型实验需要耗费 CPU 年量级的算力;对于持续迭代、需要经验性评估每次改动的模糊测试器开发而言,这种代价在算力和时间上都难以承受。本文提出 GreenBench,一个更“绿色”的基准评测平台:与 FuzzBench 相比,它大幅增加基准程序的数量,同时大幅缩短每个模糊测试 campaign 的持续时间。实验表明,GreenBench 生成的模糊测试器排名与 FuzzBench 的结果几乎同样准确(相关性极高),但速度是后者的 18 到 61 倍。作者进一步讨论了这些发现对模糊测试社区的意义。

问题
模糊测试器评测缺乏标准化,而现有主流平台 FuzzBench 的单次实验需要 CPU 年量级的算力与时间,对频繁评估改动的模糊测试器开发而言代价过高。
方法
提出 GreenBench 基准评测平台,通过大幅增加基准程序数量并大幅缩短每个模糊测试 campaign 的时长,在显著降低资源与时间开销的同时保持评测准确性。
评估
将 GreenBench 与 FuzzBench 进行对比,衡量两者生成模糊测试器排名的相关性,并比较运行时间;结果显示排名准确性接近而速度提升 18–61 倍。
原文摘要(Abstract)

Over the last decade, fuzzing has been increasingly gaining traction due to its effectiveness in finding bugs. Nevertheless, fuzzer evaluations have been challenging during this time, mainly due to lack of standardized benchmarking. Aiming to alleviate this issue, in 2020, Google released FuzzBench, an open-source benchmarking platform, that is widely used for accurate fuzzer benchmarking. However, a typical FuzzBench experiment takes CPU years to run. If we additionally consider that fuzzers under active development evaluate any changes empirically, benchmarking becomes prohibitive both in terms of computational resources and time. In this paper, we propose GreenBench, a greener benchmarking platform that, compared to FuzzBench, significantly speeds up fuzzer evaluations while maintaining very high accuracy. In contrast to FuzzBench, GreenBench drastically increases the number of benchmarks while drastically decreasing the duration of fuzzing campaigns. As a result, the fuzzer rankings generated by GreenBench are almost as accurate as those by FuzzBench (with very high correlation), but GreenBench is from 18 to 61 times faster. We discuss the implications of these findings for the fuzzing community.

链接与引用

DOI 原文 · PDF(开放获取) · DBLP

BibTeX
@inproceedings{OunjaiWC23,
  title = {Green Fuzzer Benchmarking},
  author = {Jiradet Ounjai and Valentin Wüstholz and Maria Christakis},
  booktitle = {ISSTA},
  year = {2023},
  doi = {10.1145/3597926.3598144}
}