一句话：对神经程序平滑模糊测试器进行迄今最大规模评估，发现其性能优势不成立，并提出改进与基准测试平台。
神经程序平滑（NPS）是一类用神经网络平滑近似目标程序、以指导测试用例生成的机器学习引导模糊测试器，此前工作声称其性能优于标准灰盒模糊测试器。本文进行了目前最广泛的 NPS 模糊测试器评估，耗费超过 11 CPU 年和 5.5 GPU 年，并与标准灰盒模糊测试器对比。结果表明，NPS 模糊测试器原先的性能主张无法成立，这一差距与已有工作在基础、实现和实验设置上的局限有关。作者首次深入分析 NPS 中机器学习和基于梯度变异各自的贡献，实现 Neuzz++，说明解决 NPS 的实际局限能提升性能，但标准灰盒模糊测试器几乎总能超越基于 NPS 的模糊测试器。基于此，论文提出面向机器学习模糊测试基准测试的新指南，并发布 MLFuzz 平台以支持可复现评估。
Testing with randomly generated inputs (fuzzing) has gained significant traction due to its capacity to expose program vulnerabilities automatically. Fuzz testing campaigns generate large amounts of data, making them ideal for the application of machine learning (ML). Neural program smoothing, a specific family of ML-guided fuzzers, aims to use a neural network as a smooth approximation of the program target for new test case generation. In this paper, we conduct the most extensive evaluation of neural program smoothing (NPS) fuzzers against standard gray-box fuzzers (>11 CPU years and >5.5 GPU years), and make the following contributions: We find that the original performance claims for NPS fuzzers do not hold; a gap we relate to fundamental, implementation, and experimental limitations of prior works. We contribute the first in-depth analysis of the contribution of machine learning and gradient-based mutations in NPS. We implement Neuzz++, which shows that addressing the practical limitations of NPS fuzzers improves performance, but that standard gray-box fuzzers almost always surpass NPS-based fuzzers. As a consequence, we propose new guidelines targeted at benchmarking fuzzing based on machine learning, and present MLFuzz, a platform with GPU access for easy and reproducible evaluation of ML-based fuzzers. Neuzz++, MLFuzz, and all our data are public.