GTC Silicon Valley-2019 ID:S9414:Meta-Optimization on a Distributed System for Deep Reinforcement Learning
Iuri Frosio(NVIDIA),Gregory Heinrich(NVIDIA)
Training intelligent agents with reinforcement learning is a notoriously unstable process. Although massive parallelization on GPUs and distributed systems can reduce instabilities, the success of training remains strongly influenced by the choice of hyperparameters. We'll describe a novel meta-optimization algorithm for distributed systems that solves a set of optimization problems in parallel while looking for the optimal hyperparameters. We'll also show how it applies to deep reinforcement learning. We'll demonstrate how the algorithm can fine-tune hyperparameters while learning to play different Atari games. Compared with existing approaches, our algorithm releases more computational resources during training by means of a stochastic scheduling procedure. Our algorithm has been implemented on top of MagLev, the NVIDIA AI training and inference infrastructure.