一句话：首个面向 npm 服务端 JavaScript 的可执行漏洞基准套件，含 600 个漏洞利用与验证 oracle。
npm 生态系统漏洞频发，但缺乏真实、带可执行利用和修复信息的可复用漏洞基准，导致研究者各自构建评估数据集、难以横向比较。本文提出 SecBench.js，首个服务端 JavaScript 漏洞基准套件，包含 600 个漏洞，覆盖五类最常见漏洞，每个漏洞配有利用 payload 与验证利用是否成功的 oracle。基于该基准展示三类应用：与公开安全公告交叉核对，发现 19 个包中 168 个被错误标注的易受攻击版本；对 exploit 施加简单代码变换可识别有缺陷的修复；动态分析 exec() 等 sink API 调用得到代码位置 ground truth 以评估漏洞检测器。此外还发现 20 个零日漏洞，多数已被从业者确认。
NPM is the largest software ecosystem in the world, offering millions of free, reusable packages. In recent years, various security threats to packages published on npm have been reported, including vulnerabilities that affect millions of users. To continuously improve techniques for detecting vulnerabilities and mitigating attacks that exploit them, a reusable benchmark of vulnerabilities would be highly desirable. Ideally, such a benchmark should be realistic, come with executable exploits, and include fixes of vulnerabilities. Unfortunately, there currently is no such benchmark, forcing researchers to repeatedly develop their own evaluation datasets and making it difficult to compare techniques with each other. This paper presents SecBench.js,, the first comprehensive benchmark suite of vulnerabilities and executable exploits for npm. The benchmark comprises 600 vulnerabilities, which cover the five most common vulnerability classes for server-side JavaScript. Each vulnerability comes with a payload that exploits the vulnerability and an oracle that validates successful exploitation. SecBench.js, enables various applications, of which we explore three in this paper: (i) cross-checking SecBench.js, against public security advisories reveals 168 vulnerable versions in 19 packages that are mislabeled in the advisories; (ii) applying simple code transformations to the exploits in our suite helps identify flawed fixes of vulnerabilities; (iii) dynamically analyzing calls to common sink APIs, e.g., exec(), yields a ground truth of code locations for evaluating vulnerability detectors. Beyond providing a reusable benchmark to the community, our work identified 20 zero-day vulnerabilities, most of which are already acknowledged by practitioners.
@inproceedings{BhuiyanPVPS23,
title = {SecBench.js: An Executable Security Benchmark Suite for Server-Side JavaScript},
author = {Masudul Hasan Masud Bhuiyan and Adithya Srinivas Parthasarathy and Nikos Vasilakis and Michael Pradel and Cristian-Alexandru Staicu},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse48619.2023.00096}
}