Applying supervised and reinforcement learning methods to create neural-network-based agents for playing StarCraft II
Authors:
MichaĆ Opanowicz
Abstract:
Recently, multiple approaches for creating agents for playing various complex real-time computer games such as StarCraft II or Dota 2 were proposed, however, they either embed a significant amount of expert knowledge into the agent or use a prohibitively large for most researchers amount of computational resources. We propose a neural network architecture for playing the full two-player match of S…
▽ More
Recently, multiple approaches for creating agents for playing various complex real-time computer games such as StarCraft II or Dota 2 were proposed, however, they either embed a significant amount of expert knowledge into the agent or use a prohibitively large for most researchers amount of computational resources. We propose a neural network architecture for playing the full two-player match of StarCraft II trained with general-purpose supervised and reinforcement learning, that can be trained on a single consumer-grade PC with a single GPU. We also show that our implementation achieves a non-trivial performance when compared to the in-game scripted bots. We make no simplifying assumptions about the game except for playing on a single chosen map, and we use very little expert knowledge. In principle, our approach can be applied to any RTS game with small modifications. While our results are far behind the state-of-the-art large-scale approaches in terms of the final performance, we believe our work can serve as a solid baseline for other small-scale experiments.
△ Less
Submitted 26 September, 2021;
originally announced September 2021.
Towards robust and domain agnostic reinforcement learning competitions
Authors:
William Hebgen Guss,
Stephanie Milani,
Nicholay Topin,
Brandon Houghton,
Sharada Mohanty,
Andrew Melnik,
Augustin Harter,
Benoit Buschmaas,
Bjarne Jaster,
Christoph Berganski,
Dennis Heitkamp,
Marko Henning,
Helge Ritter,
Chengjie Wu,
Xiaotian Hao,
Yiming Lu,
Hangyu Mao,
Yihuan Mao,
Chao Wang,
Michal Opanowicz,
Anssi Kanervisto,
Yanick Schraner,
Christian Scheller,
Xiren Zhou,
Lu Liu
, et al. (4 additional authors not shown)
Abstract:
Reinforcement learning competitions have formed the basis for standard research benchmarks, galvanized advances in the state-of-the-art, and shaped the direction of the field. Despite this, a majority of challenges suffer from the same fundamental problems: participant solutions to the posed challenge are usually domain-specific, biased to maximally exploit compute resources, and not guaranteed to…
▽ More
Reinforcement learning competitions have formed the basis for standard research benchmarks, galvanized advances in the state-of-the-art, and shaped the direction of the field. Despite this, a majority of challenges suffer from the same fundamental problems: participant solutions to the posed challenge are usually domain-specific, biased to maximally exploit compute resources, and not guaranteed to be reproducible. In this paper, we present a new framework of competition design that promotes the development of algorithms that overcome these barriers. We propose four central mechanisms for achieving this end: submission retraining, domain randomization, desemantization through domain obfuscation, and the limitation of competition compute and environment-sample budget. To demonstrate the efficacy of this design, we proposed, organized, and ran the MineRL 2020 Competition on Sample-Efficient Reinforcement Learning. In this work, we describe the organizational outcomes of the competition and show that the resulting participant submissions are reproducible, non-specific to the competition environment, and sample/resource efficient, despite the difficult competition task.
△ Less
Submitted 7 June, 2021;
originally announced June 2021.