一句话：提出 ReMoS，通过基于神经元覆盖的相关模型切片，在迁移学习微调中减少教师模型缺陷继承。
迁移学习允许开发者基于预训练教师模型构建自定义学生模型，但教师模型中的对抗脆弱性、后门等缺陷也可能被学生继承。减少此类缺陷很困难，因为学生模型通常不知道教师模型如何训练或是否被攻击。本文提出 ReMoS，一种相关模型切片技术，用于在迁移学习中减少缺陷继承并保留教师模型中有用知识。ReMoS 通过在学生任务上剖析教师模型获得神经元覆盖信息，计算与任务相关的模型切片（模型权重的子集），仅用该切片微调学生模型，而对无关权重从头重训练，以降低继承缺陷的风险。实验覆盖 7 种 DNN 缺陷、4 个 DNN 模型和 8 个数据集，结果表明 ReMoS 能有效减少继承缺陷：CV 任务降低 63% 至 86%，NLP 任务降低 40% 至 61%，同时平均仅损失约 3% 的准确率。
Transfer learning is a popular software reuse technique in the deep learning community that enables developers to build custom models (students) based on sophisticated pretrained models (teachers). However, like vulnerability inheritance in traditional software reuse, some defects in the teacher model may also be inherited by students, such as well-known adversarial vulnerabilities and backdoors. Reducing such defects is challenging since the student is unaware of how the teacher is trained and/or attacked. In this paper, we propose ReMoS, a relevant model slicing technique to reduce defect inheritance during transfer learning while retaining useful knowledge from the teacher model. Specifically, ReMoS computes a model slice (a subset of model weights) that is relevant to the student task based on the neuron coverage information obtained by profiling the teacher model on the student task. Only the relevant slice is used to finetune the student model, while the irrelevant weights are retrained from scratch to minimize the risk of inheriting defects. Our experiments on seven DNN defects, four DNN models, and eight datasets demonstrate that ReMoS can reduce inherited defects effectively (by 63% to 86% for CV tasks and by 40% to 61% for NLP tasks) and efficiently with minimal sacrifice of accuracy (3% on average).