一句话：提出无需重训练与数据、不牺牲精度的集成策略，通过对齐新旧模型的不确定性来减少 DNN 回归错误。
DNN 回归错误指旧版本模型预测正确而新版本模型预测错误的情形，这在生产系统中升级模型时频繁发生，会造成不成比例的用户体验退化。本文提出一种轻量级的回归错误削减方法，目标有二：一是不需要模型重训练、甚至不需要数据；二是不牺牲整体精度。该方法的核心洞见在于 DNN 模型固有但长期未被管理的不确定性，作者据此提出一个简单而有效的集成策略，对新旧两个模型的不确定性进行估计与对齐。文中从理论上证明该策略能够实现帕累托改进，即在不损害整体精度的前提下减少回归错误，并在实践中基本达成这一目标。在多种代表性模型与数据集上的全面实验表明，该方法显著优于现有最先进的替代方案。
Regression errors of Deep Neural Network (DNN) models refer to the case that predictions were correct by the old-version model but wrong by the new-version model. They frequently occur when upgrading DNN models in production systems, causing disproportionate user experience degradation. In this paper, we propose a lightweight regression error reduction approach with two goals: 1) requiring no model retraining and even data, and 2) not sacrificing the accuracy. The proposed approach is built upon the key insight rooted in the unmanaged model uncertainty, which is intrinsic to DNN models, but has not been thoroughly explored especially in the context of quality assurance of DNN models. Specifically, we propose a simple yet effective ensemble strategy that estimates and aligns the two models' uncertainty. We show that a Pareto improvement that reduces the regression errors without compromising the overall accuracy can be guaranteed in theory and largely achieved in practice. Comprehensive experiments with various representative models and datasets confirm that our approaches significantly outperform the state-of-the-art alternatives.
@inproceedings{LiZXYCCML23,
title = {Lightweight Approaches to DNN Regression Error Reduction: An Uncertainty Alignment Perspective},
author = {Zenan Li and Maorun Zhang and Jingwei Xu and Yuan Yao and Chun Cao and Taolue Chen and Xiaoxing Ma and Jian Lü},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse48619.2023.00106}
}