一句话：提出 αNAS：在程序属性抽象空间中搜索神经架构，并用合成过程具体化为网络，显著减少参数与计算量。
神经架构搜索（NAS）是深度学习的重要工具，但现有方法多局限于高度结构化的设计空间，只能探索全部架构空间的一小部分，且依赖专家手工设计。本文提出在程序属性的抽象空间中进行 NAS：该空间更小，且属性相似的架构性能相近，因此搜索更高效。为将属性具体化为网络，作者引入高效合成过程，并在进化框架中以程序属性引导变异，实现 αNAS。以 ResNet-34 为起点，在 CIFAR-10 上精度略升且参数量减少 96%；在 ImageNet 上相较 Vision Transformer、ResNet-50 和 EfficientNet 分别减少 30%、23%/14%、7% 的 FLOPS/参数，精度不降。
Neural architecture search (NAS) has become an increasingly important tool within the deep learning community in recent years, yielding many practical advancements in the design of deep neural network architectures. However, most existing approaches operate within highly structured design spaces, and hence (1) explore only a small fraction of the full search space of neural architectures while also (2) requiring significant manual effort from domain experts. In this work, we develop techniques that enable efficient NAS in a significantly larger design space. In particular, we propose to perform NAS in an abstract search space of program properties. Our key insights are as follows: (1) an abstract search space can be significantly smaller than the original search space, and (2) architectures with similar program properties should also have similar performance; thus, we can search more efficiently in the abstract search space. To enable this approach, we also introduce a novel efficient synthesis procedure, which performs the role of concretizing a set of promising program properties into a satisfying neural architecture. We implement our approach, αNAS, within an evolutionary framework, where the mutations are guided by the program properties. Starting with a ResNet-34 model, αNAS produces a model with slightly improved accuracy on CIFAR-10 but 96% fewer parameters. On ImageNet, αNAS is able to improve over Vision Transformer (30% fewer FLOPS and parameters), ResNet-50 (23% fewer FLOPS, 14% fewer parameters), and EfficientNet (7% fewer FLOPS and parameters) without any degradation in accuracy.
@article{paperbot1446,
title = {Neural architecture search using property guided synthesis},
author = {Charles Jin and Phitchaya Mangpo Phothilimthana and Sudip Roy},
journal = {Proceedings of the ACM on Programming Languages},
volume = {6},
number = {OOPSLA2},
year = {2022},
doi = {10.1145/3563329}
}