一句话:用大语言模型嵌入向量结合聚类对模糊测试崩溃输入去重,效果显著优于手工栈迹比较与现有SOTA方法。
模糊测试能高效发现软件漏洞,但一场测试活动往往产生大量崩溃输入,其中许多源于同一底层缺陷,人工分析成本极高,因此崩溃去重十分关键。现有去重方法多依赖栈迹比较或其他崩溃时采集的信息,虽然提出了多种相似度度量,但去重效果常常不理想。本文提出 GPTrace:一种利用大语言模型对与崩溃相关的多种数据源计算嵌入向量,并将这些嵌入作为聚类算法输入来评估崩溃相似性的去重工作流。作者在来自 14 个不同目标、50 个真实标签的 30 万余个崩溃输入上进行了评估。结果显示,GPTrace 的去重效果相较手工设计的栈迹比较方法有明显提升,并且优于更复杂但灵活性较差的最新方法。
Fuzzing is a highly effective method for uncovering software vulnerabilities, but analyzing the resulting data typically requires substantial manual effort. This is amplified by the fact that fuzzing campaigns often find a large number of crashing inputs, many of which share the same underlying bug. Crash deduplication is the task of finding such duplicate crashing inputs and thereby reducing the data that needs to be examined. Many existing deduplication approaches rely on comparing stack traces or other information that is collected when a program crashes. Although various metrics for measuring the similarity of such pieces of information have been proposed, many do not yield satisfactory deduplication results. In this work, we present GPTrace, a deduplication workflow that leverages a large language model to evaluate the similarity of various data sources associated with crashes by computing embedding vectors and supplying those as input to a clustering algorithm. We evaluate our approach on over 300000 crashing inputs belonging to 50 ground truth labels from 14 different targets. The deduplication results produced by GPTrace show a noticeable improvement over hand-crafted stack trace comparison methods and even more complex state-of-the-art approaches that are less flexible.