一句话：通过大规模基准评测揭示神经程序平滑模糊测试的不稳定性，并提出 PreFuzz 显著提升其边覆盖率。
模糊测试常被建模为以覆盖率最大化为目标的优化问题，但传统搜索式方法易导致变异低效。近期提出的 Neuzz 与 MTFuzz 利用神经网络近似程序分支行为，并以梯度较大的字节进行变异。然而二者仅在有限数据集上评测，其关键组件与实际影响因素尚不明确。本文首先构建包含 28 个流行开源项目的大规模基准套件，对 Neuzz 与 MTFuzz 展开广泛评测，发现其边覆盖率表现不稳定，神经网络模型与变异策略均无法持续有效，梯度引导机制的作用被削弱。基于此，作者提出简洁技术 PreFuzz：以资源高效的边选择机制增强梯度引导，并用概率字节选择机制提升变异有效性。评测表明 PreFuzz 能显著提升 Neuzz/MTFuzz 的边覆盖率，并总结出若干推进未来研究的实践指南。
Fuzzing nowadays has been commonly modeled as an optimization problem, e.g., maximizing code coverage under a given time budget via typical search-based solutions such as evolutionary algorithms. However, such solutions are widely argued to cause inefficient computing resource usage, i.e., inefficient mutations. To address this issue, two neural program-smoothing-based fuzzers, Neuzz and MTFuzz, have been recently proposed to approximate program branching behaviors via neural network models, which input byte sequences of a seed and output vectors representing program branching behaviors. Moreover, assuming that mutating the bytes with larger gradients can better explore branching behaviors, they develop strategies to mutate such bytes for generating new seeds as test cases. Meanwhile, although they have been shown to be effective in the original papers, they were only evaluated upon a limited dataset. In addition, it is still unclear how their key technical components and whether other factors can impact fuzzing performance. To further investigate neural program-smoothing-based fuzzing, we first construct a large-scale benchmark suite with a total of 28 popular open-source projects. Then, we extensively evaluate Neuzz and MTFuzz on such benchmarks. The evaluation results suggest that their edge coverage performance can be unstable. Moreover, neither neural network models nor mutation strategies can be consistently effective, and the power of their gradient-guidance mechanisms have been compromised. Inspired by such findings, we propose a simplistic technique, PreFuzz, which improves neural program-smoothing-based fuzzers with a resource-efficient edge selection mechanism to enhance their gradient guidance and a probabilistic byte selection mechanism to further boost mutation effectiveness. Our evaluation results indicate that PreFuzz can significantly increase the edge coverage of Neuzz/MTFuzz, and also reveal multiple practical guidelines to advance future research on neural program-smoothing-based fuzzing.
@inproceedings{WuJXZ0MNWCZ22,
title = {Evaluating and improving neural program-smoothing-based fuzzing},
author = {Mingyuan Wu and Ling Jiang and Jiahong Xiang and Yuqun Zhang and Guowei Yang and Huixin Ma and Sen Nie and Shi Wu and Heming Cui and Lingming Zhang},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510089}
}