-
Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces
Authors:
Craig J. Bester,
Steven D. James,
George D. Konidaris
Abstract:
Parameterised actions in reinforcement learning are composed of discrete actions with continuous action-parameters. This provides a framework for solving complex domains that require combining high-level actions with flexible control. The recent P-DQN algorithm extends deep Q-networks to learn over such action spaces. However, it treats all action-parameters as a single joint input to the Q-networ…
▽ More
Parameterised actions in reinforcement learning are composed of discrete actions with continuous action-parameters. This provides a framework for solving complex domains that require combining high-level actions with flexible control. The recent P-DQN algorithm extends deep Q-networks to learn over such action spaces. However, it treats all action-parameters as a single joint input to the Q-network, invalidating its theoretical foundations. We analyse the issues with this approach and propose a novel method, multi-pass deep Q-networks, or MP-DQN, to address them. We empirically demonstrate that MP-DQN significantly outperforms P-DQN and other previous algorithms in terms of data efficiency and converged policy performance on the Platform, Robot Soccer Goal, and Half Field Offense domains.
△ Less
Submitted 10 May, 2019;
originally announced May 2019.
-
Dynamics of Ion Temperature Gradient Turbulence and Transport with a Static Magnetic Island
Authors:
Olivier Izacard,
Christopher Holland,
Spencer D. James,
Dylan P. Brennan
Abstract:
Understanding the interaction mechanisms between large-scale magnetohydrodynamic instabilities and small-scale drift-wave microturbulence is essential for predicting and optimizing the performance of magnetic confinement based fusion energy experiments. We report progress on understanding these interactions using both analytic theory and numerical simulations performed with the BOUT++ [B. Dudson e…
▽ More
Understanding the interaction mechanisms between large-scale magnetohydrodynamic instabilities and small-scale drift-wave microturbulence is essential for predicting and optimizing the performance of magnetic confinement based fusion energy experiments. We report progress on understanding these interactions using both analytic theory and numerical simulations performed with the BOUT++ [B. Dudson et al., Comput. Phys. Comm. 180, 1467 (2009)] framework. This work focuses upon the dynamics of the ion temperature gradient instability in the presence of a background static magnetic island, using a weakly electromagnetic two-dimensional five-field fluid model. It is found that the island width must exceed a threshold size (comparable to the turbulent correlation length in the no-island limit) to significantly impact the turbulence dynamics, with the primary impact being an increase in turbulent fluctuation and heat flux amplitudes. The turbulent radial ion energy flux is shown to localize near the X-point, but does so asymmetrically in the poloidal dimension. An effective turbulent resistivity which acts upon the island outer layer is also calculated, and shown to always be significantly (10x - 100x) greater than the collisional resistivity used in the simulations.
△ Less
Submitted 12 September, 2015; v1 submitted 24 June, 2015;
originally announced June 2015.
-
Line profile variations in $γ$ Doradus
Authors:
L. A. Balona,
T. Böhm,
B. H. Foing,
K. K. Ghosh,
E. Janot-Pacheco,
K. Krisciunas,
A-M Lagrange,
W. A. Lawson,
S. D. James,
J. Baudrand,
C. Catala,
M. Dreux,
P. Felenbok,
J. B. Hearnshaw
Abstract:
We present data from high-dispersion echelle spectra and simultaneous $uvby$ photometry for $γ$~Doradus. These data were obtained from several sites during 1994 November as part of the MUSICOS-94 campaign. The star has two closely-spaced periods of about 0.75 d and is the brightest member of a new class of variable early F-type stars. A previously suspected third period, very close to the other…
▽ More
We present data from high-dispersion echelle spectra and simultaneous $uvby$ photometry for $γ$~Doradus. These data were obtained from several sites during 1994 November as part of the MUSICOS-94 campaign. The star has two closely-spaced periods of about 0.75 d and is the brightest member of a new class of variable early F-type stars. A previously suspected third period, very close to the other two, is confirmed. Previous observations indicated that sudden changes could be expected in the spectrum, but none were found during the campaign. The radial velocities rule out the possibility of a close companion. The phasing between the radial velocity and light curve of the strongest periodic component rules out the starspot model. The only viable mechanism for understanding the variability is nonradial pulsation. We used the method of moments to identify the modes of pulsation of the three periodic components. These appear to be sectorial retrograde modes with spherical harmonic degrees, ($\ell, m$), as follows: $f_1$ = (3,3), $f_2$ = (1,1) and $f_4$ = (1,1). The angle of inclination of the star is found to be $i \approx 70^\circ$.
△ Less
Submitted 11 March, 1996;
originally announced March 1996.
