Massively Parallel Methods for Deep Reinforcement Learning
Authors:
Arun Nair,
Praveen Srinivasan,
Sam Blackwell,
Cagdas Alcicek,
Rory Fearon,
Alessandro De Maria,
Vedavyas Panneershelvam,
Mustafa Suleyman,
Charles Beattie,
Stig Petersen,
Shane Legg,
Volodymyr Mnih,
Koray Kavukcuoglu,
David Silver
Abstract:
We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience. We used our architecture to implement the…
▽ More
We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience. We used our architecture to implement the Deep Q-Network algorithm (DQN). Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment, using identical hyperparameters. Our performance surpassed non-distributed DQN in 41 of the 49 games and also reduced the wall-time required to achieve these results by an order of magnitude on most games.
△ Less
Submitted 16 July, 2015; v1 submitted 15 July, 2015;
originally announced July 2015.
