一句话：提出基于FPGA的A3C深度强化学习平台FA3C，以单精度浮点同时支持推理与训练，性能和能效均超越GPU。
深度强化学习（Deep RL）常借助深度神经网络（DNN）避免手工特征工程，但训练与推理计算开销大。A3C是代表性异步深度强化学习方法。传统FPGA DNN加速器多面向定点推理，不支持训练。本文提出FA3C，一个基于FPGA的A3C深度强化学习平台，采用单精度浮点同时支持推理和训练，并在多个A3C agent上学习六款Atari 2600游戏的控制策略。实验表明，FA3C性能超过高端GPU平台NVIDIA Tesla P100，比当时最优GPU实现快27.9%，能效高1.62倍。
Deep Reinforcement Learning (Deep RL) is applied to many areas where an agent learns how to interact with the environment to achieve a certain goal, such as video game plays and robot controls. Deep RL exploits a DNN to eliminate the need for handcrafted feature engineering that requires prior domain knowledge. The Asynchronous Advantage Actor-Critic (A3C) is one of the state-of-the-art Deep RL methods. In this paper, we present an FPGA-based A3C Deep RL platform, called FA3C. Traditionally, FPGA-based DNN accelerators have mainly focused on inference only by exploiting fixed-point arithmetic. Our platform targets both inference and training using single-precision floating-point arithmetic. We demonstrate the performance and energy efficiency of FA3C using multiple A3C agents that learn the control policies of six Atari 2600 games. Its performance is better than a high-end GPU-based platform (NVIDIA Tesla P100). FA3C achieves 27.9% better performance than that of a state-of-the-art GPU-based implementation. Moreover, the energy efficiency of FA3C is 1.62x better than that of the GPU-based implementation.
DOI 原文 ·
@inproceedings{paperbot4924,
title = {FA3C},
author = {Hyungmin Cho and Pyeongseok Oh and Jiyoung Park and Wookeun Jung and Jaejin Lee},
booktitle = {ASPLOS},
year = {2019},
doi = {10.1145/3297858.3304058}
}