一句话：提出 DeepArc，将神经网络拆分为语义相似的连续模块，从而降低模型重构与再适配等维护成本。
神经网络通常以单体形式交付，不像传统软件那样由可分解模块组成，导致模型重构、再适配等维护任务困难。本文提出 DeepArc，一种神经网络模块化方法：将网络分解为若干连续模块，每个模块封装语义相似的连续层。该模块化支持只剪枝和微调少量局部神经元与层，即可重构模型以保持已有特征（如模型压缩），或增强模型以拟合新样本。实验表明，DeepArc 使当前先进模型压缩技术的运行时效率提升 14.8%；与传统重训练相比，拟合对抗样本和修复表现不佳模型时平均只需训练不到 20% 的神经元，训练速度提升 32.85%，并达到相近的预测性能。
Neural networks are an emerging data-driven programming paradigm widely used in many areas. Unlike traditional software systems consisting of decomposable modules, a neural network is usually delivered as a monolithic package, raising challenges for some maintenance tasks such as model restructure and re-adaption. In this work, we propose DeepArc, a novel modularization method for neural networks, to reduce the cost of model maintenance tasks. Specifically, DeepArc decomposes a neural network into several consecutive modules, each of which encapsulates consecutive layers with similar semantics. The network modularization facilitates practical tasks such as refactoring the model to preserve existing features (e.g., model compression) and enhancing the model with new features (e.g., fitting new samples). The modularization and encapsulation allow us to restructure or retrain the model by only pruning and tuning a few localized neurons and layers. Our experiments show that (1) DeepArc can boost the runtime efficiency of the state-of-the-art model compression techniques by 14.8%; (2) compared to the traditional model retraining, DeepArc only needs to train less than 20% of the neurons on average to fit adversarial samples and repair under-performing models, leading to 32.85% faster training performance while achieving similar model prediction performance.
@inproceedings{RenLXLSFD23,
title = {DeepArc: Modularizing Neural Networks for the Model Maintenance},
author = {Xiaoning Ren and Yun Lin and Yinxing Xue and Ruofan Liu and Jun Sun and Zhiyong Feng and Jin Song Dong},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse48619.2023.00092}
}