一句话：提出训练中模块化（MwT），将模块化融入 DNN 训练，以损失函数优化内聚与耦合，提升模型复用效率。
深度神经网络训练成本高昂，复用已有模型成为重要思路。但整模型复用会带来额外开销或继承无用功能，现有“训练后模块化”因模型并非为模块化而训练，易导致较大开销与精度损失。本文提出“训练中模块化”（MwT），将模块化嵌入训练过程，通过两个损失函数分别优化模块内聚性和模块间耦合性，使 CNN 模型在训练后具有结构化模块。在代表性模型上的评估表明，MwT 优于最先进的训练后模块化方法：精度损失仅 1.13 个百分点，模块核保留率仅 14.58%，较基线降低 74.31%，训练与模块化总耗时 108 分钟，约为基线一半。该工作为 DNN 模型模块化与复用提供了新范式。
Deep neural network (DNN) models have become increasingly crucial components of intelligent software systems. However, training a DNN model is typically expensive in terms of both time and computational resources. To address this issue, recent research has focused on reusing existing DNN models - borrowing the concept of software reuse in software engineering. However, reusing an entire model could cause extra overhead or inherit the weaknesses from the undesired functionalities. Hence, existing work proposes to decompose an already trained model into modules, i.e., modularizing-after-training, to enable module reuse. Since the trained models are not built for modularization, modularizing-after-training may incur huge overhead and model accuracy loss. In this paper, we propose a novel approach that incorporates modularization into the model training process, i.e., modularizing-while-training (MwT). We train a model to be structurally modular through two loss functions that optimize intra-module cohesion and inter-module coupling. We have implemented the proposed approach for modularizing Convolutional Neural Network (CNN) models. The evaluation results on representative models demonstrate that MwT outperforms the existing state-of-the-art modularizing-after-training approach. Specifically, the accuracy loss caused by MwT is only 1.13 percentage points, which is less than that of the existing approach. The kernel retention rate of the modules generated by MwT is only 14.58%, with a reduction of 74.31% over the existing approach. Furthermore, the total time cost required for training and modularizing is only 108 minutes, which is half the time required by the existing approach. Our work demonstrates that MwT is a new and more effective paradigm for realizing DNN model modularization, offering a fresh perspective on achieving model reuse.
DOI 原文 ·
@inproceedings{paperbot11465,
title = {Modularizing while Training: A New Paradigm for Modularizing DNN Models},
author = {Binhang Qi and Hailong Sun and Hongyu Zhang and Ruobing Zhao and Xiang Gao},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3608135}
}