一句话:提出认证持续学习,在再训练神经网络时尽可能保持已认证的正确性,并兼顾效用。
神经网络验证近年取得进展,使网络正确性可被认证;但实际网络常因新数据分布或新任务而持续再训练,再训练可能破坏已验证性质,灾难性遗忘尤甚。本文提出“认证持续学习”,在采用现有持续学习方法更新回归网络时,尽量保留已验证的正确性属性。方法将认证约束融入持续学习过程,以在适应新数据的同时避免破坏已证明区域。作者在多个神经网络和两种持续学习方法上进行评估,结果显示该方法高效,训练后的模型能保持已认证的正确性,且通常维持较高效用。
On the one hand, there has been considerable progress on neural network verification in recent years, which makes certifying neural networks a possibility. On the other hand, neural networks in practice are often re-trained over time to cope with new data distribution or for solving different tasks (a.k.a. continual learning). Once re-trained, the verified correctness of the neural network is likely broken, particularly in the presence of the phenomenon known as catastrophic forgetting. In this work, we propose an approach called certified continual learning which improves existing continual learning methods by preserving, as long as possible, the established correctness properties of a verified network. Our approach is evaluated with multiple neural networks and on two different continual learning methods. The results show that our approach is efficient and the trained models preserve their certified correctness and often maintain high utility.