一句话：提出开源工具 DatumAid 与 OrchesTrain，用于标注数据集管理和深度模型训练自动化，并提升实验可复现性。
深度模型训练依赖大规模数据集与大量超参数/配置的受控实验，单次训练可能耗时数天，因此数据管理、配置正确性与实验可复现性至关重要。本文提出两个开源工具：DatumAid 集成 CVAT，支持筛选标注数据、操作与导出数据集，并以配置文件提供灵活性；OrchesTrain 自动化模型训练，可接入 PyTorch 模型，支持 Wandb、MLflow、TensorBoard 日志器，并将配置、代码和序列化模型以 YAML 结构保存以保证复现。两工具已在 GitHub 公开。
Designing and optimizing deep models require managing large datasets and conducting carefully designed controlled experiments that depend on large sets of hyper-parameters and problem dependent software/data configurations. These experiments are executed by training the model under observation with varying configurations. Since executing a typical training run can take days even on proven acceleration fabrics such as Graphics Processing Units (GPU), properly managing training data, avoiding human error in configuration preparations and securing the repeatability of the experiments are of utmost importance. In this paper, we present two open source software tools that aim to achieve these goals, namely, a Dataset Manager (DatumAid) tool and a Training Automation Manager (OrchesTrain) tool. DatumAid is a software tool that integrates with Computer Vision Annotation Tool (CVAT) to facilitate the management of annotated datasets. By adding additional functionality, DatumAid allows users to filter labeled data, manipulate datasets, and export datasets for training purposes. The tool adopts a simple code structure while providing flexibility to users through configuration files. OrchesTrain aims to automate model training process by facilitating rapid preparation and training of models in the desired style for the intended tasks. Users can seamlessly integrate their models prepared in the PyTorch library into the system and leverage the full capabilities of OrchesTrain. It enables the simultaneous or separate usage of Wandb, MLflow, and TensorBoard loggers. To ensure reproducibility of the conducted experiments, all configurations and codes are saved to the selected logger in an appropriate structure within a YAML file along with the serialized model files. Both software tools are publicly available on GitHub.
@inproceedings{TirasogluTEYBA23,
title = {Open Source Software Tools for Data Management and Deep Model Training Automation},
author = {Umut Tıraşoğlu and Abdussamet Türker and Adnan Ekici and Hayri Yiğit and Yusuf Enes Bölükbaşı and Toygar Akgün},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00014}
}