一句话：提出 H2O-NAS，用大规模并行 one-shot 搜索与硬件优化搜索空间自动设计高精度、高性能模型。
随着机器学习模型与算力需求持续增长，如何自动设计既精确又能在异构硬件上高效运行的模型成为关键问题。本文提出 H2O-NAS，面向超大规模硬件优化的神经架构搜索。其核心包括：可扩展到 O(10^280) 搜索空间并处理大规模生产流量的新型大规模并行“one-shot”搜索算法与智能权重共享；针对异构硬件上多种 ML 模型定制的硬件优化搜索空间；以及面向大规模部署的新型两阶段混合性能模型和多目标奖励函数。摘要未报告具体精度或加速比，但强调该框架能自动产生适配底层硬件的高精度、高性能模型，并支撑生产级流量与部署。
Recent advances in machine learning have leveraged dramatic increases in computational power, a trend expected to continue in the future. This paper introduces the first Hyperscale Hardware Optimized Neural Architecture Search (H2O-NAS) to automatically design accurate and performant machine learning models tailored to the underlying hardware architecture. H2O-NAS consists of three key components: a new massively parallel “one-shot” search algorithm with intelligent weight sharing, which can scale to search spaces of O(10280) and handle large volumes of production traffic; hardware-optimized search spaces for diverse ML models on heterogeneous hardware; and a novel two-phase hybrid performance model and a multi-objective reward function optimized for large scale deployments.
@inproceedings{paperbot10106,
title = {Hyperscale Hardware Optimized Neural Architecture Search},
author = {Sheng Li and Garrett Andersen and Tao Chen and Liqun Cheng and Julian Grady and Da Huang and Quoc V. Le and Andrew Li and Xin Li and Yang Li and Chen Liang and Yifeng Lu and Yun Ni and Ruoming Pang and Mingxing Tan and Martin Wicke and Gang Wu and Shengqi Zhu and Parthasarathy Ranganathan and Norman P. Jouppi},
booktitle = {ASPLOS},
year = {2023},
doi = {10.1145/3582016.3582049}
}