一句话：提出 DnnSAT，用代价模型与 SMT 求解在 AutoML 前缩减不满足资源约束的配置空间，加速 HPO 与 NAS。
深度学习模型像传统软件系统一样具有大量配置选项，包括超参数和神经网络架构。现有 AutoML 虽能自动探索配置，却未考虑内存、算力、执行时间等资源约束，训练不合规配置会导致大量试验失败、资源浪费并拖慢开发效率。本文提出 DnnSAT，一种资源引导的 AutoML 方法，将配置空间缩减形式化为约束满足问题。它包含统一解析代价模型，针对模型权重大小、浮点运算数、推理时间和 GPU 显存消耗构造约束，并利用 SMT 求解器获得可满足的超参数与架构配置。在公开基准上，DnnSAT 能加速 Hyperparameter Optimization 和 Neural Architecture Search 等先进 AutoML 方法，平均加速 1.19X 至 3.95X，并保持或提升模型学习性能。
Deep learning models, like traditional software systems, provide a large number of configuration options. A deep learning model can be configured with different hyperparameters and neural architectures. Recently, AutoML (Automated Machine Learning) has been widely adopted to automate model training by systematically exploring diverse configurations. However, current AutoML approaches do not take into consideration the computational constraints imposed by various resources such as available memory, computing power of devices, or execution time. The training with non-conforming configurations could lead to many failed AutoML trial jobs or inappropriate models, which cause significant resource waste and severely slow down development productivity. In this paper, we propose DnnSAT, a resource-guided AutoML approach for deep learning models to help existing AutoML tools efficiently reduce the configuration space ahead of time. DnnSAT can speed up the search process and achieve equal or even better model learning performance because it excludes trial jobs not satisfying the constraints and saves resources for more trials. We formulate the resource-guided configuration space reduction as a constraint satisfaction problem. DnnSAT includes a unified analytic cost model to construct common constraints with respect to the model weight size, number of floating-point operations, model inference time, and GPU memory consumption. It then utilizes an SMT solver to obtain the satisfiable configurations of hyperparameters and neural architectures. Our evaluation results demonstrate the effectiveness of DnnSAT in accelerating state-of-the-art AutoML methods (Hyperparameter Optimization and Neural Architecture Search) with an average speedup from 1.19X to 3.95X on public benchmarks. We believe that DnnSAT can make AutoML more practical in a real-world environment with constrained resources.