一句话：提出“训练控制即代码”范式与 TrAC 声明式框架，让用户用代码分析训练状态并介入控制，减少资源浪费。
训练即服务平台通常将预配置训练任务作为批处理负载运行，但配置的不可变性使训练过程中难以动态调整。现有做法依赖人工在仪表盘上监控训练进度，并采用“停止-重配置-重启”的方式，难以扩展到大量实验，还会浪费计算资源并增加调试难度。本文提出 training-control-as-code 范式，允许用户运行自定义代码来分析训练状态，并在发现异常时进行干预，从而减少资源浪费。作者实现了 TrAC 框架，提供声明式接口来声明期望的控制逻辑，并支持大规模复用。论文使用真实开源数据和模型估算 TrAC 带来的时间与资源节省，并提供了演示视频和代码。
Training-as-a-service platforms facilitate users to deploy pre-configured Generative AI training jobs as batch workloads. The immutability of configuration offers minimal flexibility to dynamically adapt to training progress. Existing approaches invariably involve manually monitoring training progress on a dashboard, and the stop-reconfigure-restart of training does not scale well with number of experiments. Relying on pre-configuration, wastes computational resources and makes debugging of training jobs difficult. We address this gap through our training-control-as-code paradigm, which allows users to run user-defined code to analyze the training state and intervene to flag anomalies and save resource wastage. Our framework TrAC offers a declarative interface to allow for declaring desired control and for reusing it at scale. Using real-world open-source data and models we provide estimates on the savings in time and resource due to TrAC. We also provide demo video: https://youtu.be/RmhBfFjd1oA and code: https://github.com/foundation-model-stack/fms-hf-tuning/blob/main/examples/trainercontroller_configs/Readme.md
DOI 原文 ·
@inproceedings{paperbot9423,
title = {Training-Control-as-Code: Towards a declarative solution to control training},
author = {Padmanabha V. Seshadri and Harikrishnan Balagopal and Mehant Kammakomati and Ashok Pon Kumar and Dushyant Behl},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00384}
}