一句话：仅用代码度量训练的分类器在漏洞发现上可与先进LLM媲美，并揭示LLM预测与代码度量存在因果依赖。
本文研究LLM在漏洞发现中进展停滞的原因。作者从经典代码度量的视角出发，比较仅用代码度量训练的分类器与最先进LLM在漏洞发现任务上的表现，并开展根因分析。令人意外的是，仅基于代码度量的分类器性能与先进LLM相当。进一步分析发现，LLM预测与代码度量之间存在强相关性和因果效应：当改变某个度量的值时，LLM预测会相应移动。这表明LLM与代码度量在相似的浅层水平上运作，限制了其捕捉复杂模式、充分发挥漏洞发现潜力的能力。基于这些发现，作者为后续研究提出了改进建议。
Large language models (LLMs) excel in many tasks of software engineering, yet progress in leveraging them for vulnerability discovery has stalled in recent years. To understand this phenomenon, we investigate LLMs through the lens of classic code metrics. Surprisingly, we find that a classifier trained solely on these metrics performs on par with state-of-the-art LLMs for vulnerability discovery. A root-cause analysis reveals a strong correlation and a causal effect between LLMs and code metrics: When the value of a metric is changed, LLM predictions tend to shift by a corresponding magnitude. This dependency suggests that LLMs operate at a similarly shallow level as code metrics, limiting their ability to grasp complex patterns and fully realize their potential in vulnerability discovery. Based on these findings, we derive recommendations on how research should more effectively address this challenge.
@inproceedings{abs-2509-19117,
title = {LLM-based Vulnerability Discovery through the Lens of Code Metrics},
author = {Felix Weissberg and Lukas Pirch and Erik Imgrund and Jonas Möller and Thorsten Eisenhofer and Konrad Rieck},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3764574}
}