一句话：首个 NPM 恶意包检测大规模实证研究：统一基准评测 11 个工具 16 个变体，并阅读源码剖析成败根因。
NPM 的开放发布模式带来日益严重的恶意包威胁，但现有检测工具各自在不同数据集与设置下评估，横向比较不可靠。本文首次对 NPM 恶意包检测开展大规模实证研究：在含 6,420 个恶意包与 7,288 个良性包的统一基准上评测 11 个工具及其 16 个变体，并阅读源码解释成败原因。主要发现：工具的精度与召回由它消解「代码能力与恶意意图」歧义的方式结构性决定，IntelGuard 取得最佳 F1 95.98%；行为共现会放大检测信号；ML 退化源于概念收敛而非概念漂移；工具组合效果等于互补性减去误报引入，最优组合达 97.21% 准确率与 97.02% F1。
The NPM ecosystem faces escalating threats from malicious packages that exploit its open publication model. While numerous detection tools have been proposed, they are evaluated on disparate datasets with inconsistent settings, making cross-tool comparison unreliable and leaving practitioners without clear guidance. We present the first large-scale empirical study of NPM malicious package detection, evaluating 11 tools with 16 variants on a unified benchmark of 6,420 malicious and 7,288 benign packages annotated with 11 behavior categories and 8 evasion techniques. Unlike prior work, we inspect each tool’s source code to explain why tools succeed or fail, not merely how often. Our key findings: (1) the precision and recall a tool achieves are structurally determined by how it resolves the ambiguity between code capability and malicious intent, with IntelGuard reaching the best F1 at 95.98% by grounding its judgment in retrieved evidence and GuardDog the best among conventional tools at 93.32%; (2) behavioral coupling amplifies detection signals when behaviors co-occur, raising SAP_DT from 3.2% to 79.3% for the collect-and-exfiltrate chain; (3) 80.3% of malware uses no evasion because the ecosystem lacks mandatory pre-publication scanning; (4) ML degradation is driven by concept convergence rather than concept drift, since malware became simpler and every decision boundary fitted to a corpus ages with it; (5) combination effectiveness equals complementarity minus false-positive introduction, not paradigm diversity. Strategic combinations reach up to 97.21% accuracy and 97.02% F1. We release our benchmark and evaluation framework.
@inproceedings{paperbot12489,
title = {How Effective Are NPM Malicious Package Detectors? A Large-Scale Empirical Study},
author = {Wenbo Guo and Zhongwen Chen and Zhengzi Xu and Chengwei Liu and Ming Kang and Shiwen Song and Chengyue Liu and Yijia Xu and Weisong Sun and Yang Liu},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3837427}
}