一句话：提出 DFuzz，让 LLM 从 API 源码检查中推理边界输入并生成测试程序，实现 DL 库 API 的全面模糊测试。
深度学习库（如 TensorFlow、PyTorch）API 数量已超 1000 个且持续增长，其中存在缓冲区溢出、释放后使用、整数溢出等漏洞，但传统模糊测试难以应对其复杂性与 API 使用模式的多样性；现有的基于 LLM 的模糊测试器又缺乏对 API 输入边界情况的深入知识，且测试输入生成效率低下。本文提出 DFuzz：一方面利用 LLM 的推理能力，从 API 代码中的检查条件出发推断可能触发错误的边界输入，并将抽取到的知识迁移到新的或很少被测试的 API 上；另一方面利用 LLM 的生成能力，高准确率地合成初始测试程序以自动化 API 测试。DFuzz 为 LLM 提供了一种新颖的 API「白盒视角」，从而实现更全面的模糊测试。实验表明 DFuzz 在 TensorFlow 与 PyTorch 上覆盖的 API 数量均超过当前最优的基于 LLM 的模糊测试器，并成功检测到 37 个缺陷，其中 8 个已修复、19 个已被开发者复现且仍在调查中。
Deep learning (DL) libraries are widely used to form the basis of various AI applications in computer vision, natural language processing, and software engineering domains. Despite their popularity, DL libraries are known to have vulnerabilities, such as buffer overflows, use-after-free, and integer overflows, that can be exploited to compromise the security or effectiveness of the underlying libraries. While traditional fuzzing techniques have been used to find bugs in software, they are not well-suited for DL libraries. In general, the complexity of DL libraries and the diversity of their APIs make it challenging to test them thoroughly. To date, mainstream DL libraries like TensorFlow and PyTorch have featured over 1,000 APIs, and the number of APIs is still growing. Fuzzing all these APIs is a daunting task, especially when considering the complexity of the input data and the diversity of the API usage patterns. Recent advances in large language models (LLMs) have illustrated the high potential of LLMs in understanding and synthesizing human-like code. Despite their high potential, we find that emerging LLM-based fuzzers are less optimal for DL library API fuzzing, given their lack of in-depth knowledge on API input edge cases and inefficiency in generating test inputs. In this paper, we propose DFuzz, a LLM-driven DL library fuzzing approach. We have two key insights: (1) With high reasoning ability, LLMs can replace human experts to reason edge cases (likely error-triggering inputs) from checks in an API's code, and transfer the extracted knowledge to test other (new or rarely-tested) APIs. (2) With high generation ability, LLMs can synthesize initial test programs with high accuracy that automates API testing. DFuzz provides LLMs with a novel “white-box view” of DL library APIs, and therefore, can leverage LLMs' reasoning and generation abilities to achieve comprehensive fuzzing. Our experimental results on popular DL libraries demonstrate that DFuzz is able to cover more APIs than SOTA (LLM-based) fuzzers on TensorFlow and PyTorch, respectively. Moreover, DFuzz successfully detected 37 bugs, with 8 already fixed and 19 replicated by the developer but still under investigation.
DOI 原文 ·
@inproceedings{paperbot8904,
title = {Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models},
author = {Kunpeng Zhang and Shuai Wang and Jitao Han and Xiaogang Zhu and Xian Li and Shaohua Wang and Sheng Wen},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00041}
}