一句话：首个针对 Hugging Face 预训练模型枢纽恶意代码投毒的系统性测量研究，并提出并落地了检测流水线 MalHug。
随着预训练模型与数据集的广泛复用，Hugging Face 等集中式模型枢纽成为协作开发的核心平台，但其上的恶意代码投毒问题日益突出。本文首次对预训练模型枢纽中的恶意代码投毒攻击开展系统性研究：通过全面的威胁分析建立模型格式分类学，并对脆弱格式进行根因分析，指出 Fickling、ModelScan 等现有工具在语义级分析与全面威胁检测上的局限。为此作者提出 MalHug——一条面向 Hugging Face 的端到端流水线，融合数据集加载脚本抽取、模型反序列化、深度污点分析与启发式模式匹配，用于检测并分类数据集与模型中的恶意代码投毒。该工具与蚂蚁集团合作部署于其内部镜像 Hugging Face 实例并运行三个月以上，累计监控超过 70.5 万个模型与 17.6 万个数据集，发现 91 个恶意模型与 9 个恶意数据集加载脚本，涉及反向 shell、浏览器凭据窃取与系统侦察等威胁，为预训练模型供应链安全提供了经工业验证的实用方案。
The proliferation of pre-trained models (PTMs) and datasets has led to the emergence of centralized model hubs like Hugging Face, which facilitate collaborative development and reuse. However, recent security reports have uncovered vulnerabilities and instances of malicious attacks within these platforms, highlighting growing security concerns. This paper presents the first systematic study of malicious code poisoning attacks on pre-trained model hubs, focusing on the Hugging Face platform. We conduct a comprehensive threat analysis, develop a taxonomy of model formats, and perform root cause analysis of vulnerable formats. While existing tools like Fickling and ModelScan offer some protection, they face limitations in semantic-level analysis and comprehensive threat detection. To address these challenges, we propose MalHug, an end-to-end pipeline tailored for Hugging Face that combines dataset loading script extraction, model deserialization, in-depth taint analysis, and heuristic pattern matching to detect and classify malicious code poisoning attacks in datasets and models. In collaboration with Ant Group, a leading financial technology company, we have implemented and deployed MalHug on a mirrored Hugging Face instance within their infrastructure, where it has been operational for over three months. During this period, MalHug has monitored more than 705K models and 176K datasets, uncovering 91 malicious models and 9 malicious dataset loading scripts. These findings reveal a range of security threats, including reverse shell, browser credential theft, and system reconnaissance. This work not only bridges a critical gap in understanding the security of the PTM supply chain but also provides a practical, industry-tested solution for enhancing the security of pre-trained model hubs.
@inproceedings{paperbot8405,
title = {Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubs},
author = {Jian Zhao and Shenao Wang and Yanjie Zhao and Xinyi Hou and Kailong Wang and Peiming Gao and Yuanchao Zhang and Chen Wei and Haoyu Wang},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695271}
}