一句话：提出将 LLM 用作边界用例生成器，构造非常规程序以弥补其倾向生成普通模式化代码的缺陷，用于深度学习库模糊测试。
深度学习库中的缺陷会影响几乎所有下游应用，因此保证其质量至关重要。为这类库做模糊测试的难点在于：输入程序必须同时满足所支持语言（如 Python）的语法/语义以及张量/算子约束，才能构造出合法的计算图。近期 TitanFuzz 表明，现代大语言模型可以被直接用来隐式学习这些语言与 DL 计算约束，从而生成合法程序。然而，LLM 受其大规模预训练语料（如 GitHub）影响，倾向于生成与常见程序模式/词元相似的「普通」程序，而模糊测试恰恰偏好覆盖边界情况、难以由人工产出的非常规输入。本文据此指出应把 LLM 当作边界用例生成器，构造非常规程序用于深度学习库模糊测试。所提供的摘要未披露具体方法与实验结果。
Bugs in Deep Learning (DL) libraries may affect almost all downstream DL applications, and it is crucial to ensure the quality of such systems. It is challenging to generate valid input programs for fuzzing DL libraries, since the input programs need to satisfy both the syntax/semantics of the supported languages (e.g., Python) and the tensor/operator constraints for constructing valid computational graphs. Recently, the TitanFuzz work demonstrates that modern Large Language Models (LLMs) can be directly leveraged to implicitly learn all the language and DL computation constraints to generate valid programs for fuzzing DL libraries (and beyond). However, LLMs tend to generate ordinary programs following similar patterns/tokens with typical programs seen in their massive pre-training corpora (e.g., GitHub), while fuzzing favors unusual inputs that cover edge cases or are unlikely to be manually produced.
DOI 原文 ·
@inproceedings{paperbot11384,
title = {Large Language Models are Edge-Case Generators: Crafting Unusual Programs for Fuzzing Deep Learning Libraries},
author = {Yinlin Deng and Chunqiu Steven Xia and Chenyuan Yang and Shizhuo Dylan Zhang and Shujing Yang and Lingming Zhang},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3623343}
}