一句话：提出 mock 深度测试方法，将 DL 工作流分解并用 mock 数据与 mock 模型对数据准备和模型设计进行独立单元测试。
深度学习已成为许多关键软件系统的组成部分，但数据与模型之间的紧耦合使得二者难以被分别测试，现有软件工程研究也较少探索这一问题。本文提出 Mock Deep Testing，一种用于深度学习应用单元测试的方法：它将 DL 工作流分解为可管理的组件，减少顺序依赖，并模块化数据准备与模型设计等关键阶段；同时用 mock 数据和 mock 模型模拟原始数据与模型的行为，以支持独立开发与测试。作者实现了面向 Keras 的框架 KUnit。在 50 个来自 Stack Overflow 和 GitHub 的 DL 程序上，mock 有效识别了数据准备阶段的 10 个问题和模型设计阶段的 53 个问题；36 名参与者的用户研究中，使用 KUnit 的被试分别解决了数据准备阶段的 25 个问题和模型设计阶段的 38 个问题。后测问卷表明 KUnit 有助于开发者独立测试各组件并有效解决问题。
While deep learning (DL) has permeated, and become an integral component of many critical software systems, today software engineering research hasn't explored how to separately test data and models that are integral for DL approaches to work effectively. The main challenge in independently testing these components arises from the tight dependency between data and models. This research explores this gap, introducing our methodology of mock deep testing for unit testing of DL applications. To enable unit testing, we introduce a design paradigm that decomposes the workflow into distinct, manageable components, minimizes sequential dependencies, and modularizes key stages of the DL, including data preparation and model design. For unit testing these components, we propose modeling their dependencies using mocks. In the context of DL, mocks refer to mock data and mock model that mimic the behavior of the original data and model, respectively. This modular approach facilitates independent development and testing of the components, ensuring comprehensive quality assurance throughout the development process. We have developed KUnit, a framework for enabling mock deep testing for the Keras library, a popular library for developing DL applications. We empirically evaluated KUnit to determine the effectiveness of mocks in independently testing data and models. Our assessment of 50 DL programs obtained from Stack Overflow and GitHub shows that mocks effectively identified 10 issues in the data preparation stage and 53 issues in the model design stage. We also conducted a user study with 36 participants using KUnit to perceive the effectiveness of our approach. Participants using KUnit successfully resolved 25 issues in the data preparation stage and 38 issues in the model design stage. Our findings highlight that mock objects provide a lightweight emulation of the dependencies for unit testing, facilitating early bug detection. Lastly, to evaluate the usability of KUnit, we conducted a post-study survey. The results reveal that KUnit is helpful to DL application developers, enabling them to independently test each component (data and model) and resolve issues effectively in different stages.
DOI 原文 ·
@inproceedings{paperbot8824,
title = {Mock Deep Testing: Toward Separate Development of Data and Models for Deep Learning},
author = {Ruchira Manke and Mohammad Wardat and Foutse Khomh and Hridesh Rajan},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00220}
}