-
Target Network and Truncation Overcome The Deadly Triad in $Q$-Learning
Authors:
Zaiwei Chen,
John Paul Clarke,
Siva Theja Maguluri
Abstract:
$Q…
▽ More
$Q$-learning with function approximation is one of the most empirically successful while theoretically mysterious reinforcement learning (RL) algorithms, and was identified in Sutton (1999) as one of the most important theoretical open problems in the RL community. Even in the basic linear function approximation setting, there are well-known divergent examples. In this work, we show that \textit{target network} and \textit{truncation} together are enough to provably stabilize $Q$-learning with linear function approximation, and we establish the finite-sample guarantees. The result implies an $O(ε^{-2})$ sample complexity up to a function approximation error. Moreover, our results do not require strong assumptions or modifying the problem parameters as in existing literature.
△ Less
Submitted 3 May, 2022; v1 submitted 4 March, 2022;
originally announced March 2022.
-
The Milky Way's middle-aged inner ring
Authors:
Shola M. Wylie,
Jonathan P. Clarke,
Ortwin E. Gerhard
Abstract:
We investigate the metallicity, age, and orbital anatomy of the inner Milky Way, specifically focussing on the outer bar region. We integrated a sample of APOGEE DR16 inner Galaxy stars in a state of the art bar-bulge potential with a slow pattern speed and investigated the link between the resulting orbits and their [Fe/H] and ages. By superimposing the orbits, we built density, [Fe/H], and age m…
▽ More
We investigate the metallicity, age, and orbital anatomy of the inner Milky Way, specifically focussing on the outer bar region. We integrated a sample of APOGEE DR16 inner Galaxy stars in a state of the art bar-bulge potential with a slow pattern speed and investigated the link between the resulting orbits and their [Fe/H] and ages. By superimposing the orbits, we built density, [Fe/H], and age maps of the inner Milky Way, which we divided further using the orbital parameters eccentricity, |Xmax|, and |Zmax|. We find that at low heights from the Galactic plane, the Galactic bar gradually transitions into a radially thick, vertically thin, elongated inner ring with average solar [Fe/H]. This inner ring is mainly composed of stars with AstroNN ages between 4 and 9 Gyr with a peak in age between 6 and 8 Gyr, making the average age of the ring ~6 Gyr. The vertical thickness of the ring decreases markedly towards younger ages. We also find very large L4 Lagrange orbits that have average solar to super-solar metallicities and intermediate ages. Lastly, we confirm a clear X-shape in the [Fe/H] and density distributions at large Galactic heights. The orbital structure obtained for the APOGEE stars reveals that the Milky Way hosts an inner ring-like structure between the planar bar and corotation. This structure is on average metal rich, intermediately aged, and enhances the horizontal metallicity gradient along the bar's major axis.
△ Less
Submitted 21 January, 2022; v1 submitted 7 October, 2021;
originally announced October 2021.
-
A2A: 21,000 bulge stars from the ARGOS survey with stellar parameters on the APOGEE scale
Authors:
S. M. Wylie,
O. E. Gerhard,
M. K. Ness,
J. P. Clarke,
K. C. Freeman,
J. Bland-Hawthorn
Abstract:
We use the data-driven method, The Cannon, to bring 21,000 stars from the ARGOS bulge survey, including 10,000 red clump stars, onto the parameter and abundance scales of the cross-Galactic survey, APOGEE, obtaining rms precisions of 0.10 dex, 0.07 dex, 74 K, and 0.18 dex for [Fe/H], [Mg/Fe], Teff, and log(g), respectively. The re-calibrated ARGOS survey - which we refer to as the A2A survey - is…
▽ More
We use the data-driven method, The Cannon, to bring 21,000 stars from the ARGOS bulge survey, including 10,000 red clump stars, onto the parameter and abundance scales of the cross-Galactic survey, APOGEE, obtaining rms precisions of 0.10 dex, 0.07 dex, 74 K, and 0.18 dex for [Fe/H], [Mg/Fe], Teff, and log(g), respectively. The re-calibrated ARGOS survey - which we refer to as the A2A survey - is combined with the APOGEE survey to investigate the abundance structure of the Galactic bulge. We find X-shaped [Fe/H] and [Mg/Fe] distributions in the bulge that are more pinched than the bulge density, a signature of its disk origin. The mean abundance along the major axis of the bar varies such that the stars are more [Fe/H]-poor and [Mg/Fe]-rich near the Galactic center than in the long bar/outer bulge region. The vertical [Fe/H] and [Mg/Fe] gradients vary between the inner bulge and long bar with the inner bulge showing a flattening near the plane that is absent in the long bar. The [Fe/H]-[Mg/Fe] distribution shows two main maxima, an ``[Fe/H]-poor [Mg/Fe]- rich'' maximum and an ``[Fe/H]-rich [Mg/Fe]-poor'' maximum, that vary in strength with position in the bulge. In particular, the outer long bar close to the Galactic plane is dominated by super-solar [Fe/H], [Mg/Fe]-normal stars. Stars composing the [Fe/H]-rich maximum show little kinematic dependence on [Fe/H], but for lower [Fe/H] the rotation and dispersion of the bulge increase slowly. Stars with [Fe/H]<-1 dex have a very different kinematic structure than stars with higher [Fe/H]. Comparing with recent models for the Galactic boxy-peanut bulge, the abundance gradients and distribution, and the relation between [Fe/H] and kinematics suggest that the stars comprising each maximum have separate disk origins with the ``[Fe/H]-poor [Mg/Fe]-rich'' stars originating from a thicker disk than the ``[Fe/H]-rich [Mg/Fe]-poor'' stars.
△ Less
Submitted 27 June, 2021;
originally announced June 2021.
-
Gas Dynamics in the Galaxy: Total Mass Distribution and the Bar Pattern Speed
Authors:
Zhi Li,
Juntai Shen,
Ortwin Gerhard,
Jonathan P. Clarke
Abstract:
Gas morphology and kinematics in the Milky Way contain key information for understanding the formation and evolution of our Galaxy. We present a high resolution hydrodynamical simulation based on a realistic barred Milky Way potential constrained by recent observations. Our model can reproduce most features in the observed longitude-velocity diagram, including the Central Molecular Zone, the Near…
▽ More
Gas morphology and kinematics in the Milky Way contain key information for understanding the formation and evolution of our Galaxy. We present a high resolution hydrodynamical simulation based on a realistic barred Milky Way potential constrained by recent observations. Our model can reproduce most features in the observed longitude-velocity diagram, including the Central Molecular Zone, the Near and Far 3-kpc arms, the Molecular Ring, and the spiral arm tangents. It can also explain the non-circular motions of masers obtained by the recent BeSSeL2 survey. The central gas kinematics are consistent with a mass of $6.9\times10^8\; {\rm M}_{\odot}$ in the Nuclear Stellar Disk. Our model predicts the formation of an elliptical gaseous ring surrounding the bar, which is composed of the 3-kpc arms, Norma arm, and the bar-spiral interfaces. This ring is similar to those "inner" rings in some Milky Way analogs with a boxy/peanut-shaped bulge. The kinematics of gas near the solar neighbourhood are governed by the Local arm, which is induced by the four major stellar spiral arms. The bar pattern speed constrained by our gas model is $37.5-40\; {\rm km}\;{\rm s}^{-1}\;{\rm kpc}^{-1}$, corresponding to a corotation radius of $R_{\rm CR}=6.0-6.4\;{\rm kpc}$. The rotation curve of our model rises gently within the central $\sim5\;{\rm kpc}$, which is significantly less steep than those predicted by modern zoom-in cosmological simulations such as Auriga.
△ Less
Submitted 10 November, 2021; v1 submitted 18 March, 2021;
originally announced March 2021.
-
The Milky Way bar/bulge in proper motions: a 3D view from VIRAC & Gaia
Authors:
Jonathan P. Clarke,
Christopher Wegg,
Ortwin Gerhard,
Leigh C. Smith,
Phil W. Lucas,
Shola M. Wylie
Abstract:
We have derived absolute proper motions of the entire Galactic bulge region from VIRAC and Gaia. We present these as both integrated on-sky maps and, after isolating standard candle red clump (RC) stars, as a function of distance using RC magnitude as a proxy. These data provide a new global, 3-dimensional view of the Milky Way barred bulge kinematics. We find a gradient in the mean longitudinal p…
▽ More
We have derived absolute proper motions of the entire Galactic bulge region from VIRAC and Gaia. We present these as both integrated on-sky maps and, after isolating standard candle red clump (RC) stars, as a function of distance using RC magnitude as a proxy. These data provide a new global, 3-dimensional view of the Milky Way barred bulge kinematics. We find a gradient in the mean longitudinal proper motion, $<μ_l^\star>$, between the different sides of the bar, which is sensitive to the bar pattern speed. The split RC has distinct proper motions and is colder than other stars at similar distance. The proper motion correlation map has a quadrupole pattern in all magnitude slices showing no evidence for a separate, more axisymmetric inner bulge component. The line-of-sight integrated kinematic maps show a high central velocity dispersion surrounded by a more asymmetric dispersion profile. $σ_{μ_l} / σ_{μ_b}$ is smallest, $\sim1.1$, near the minor axis and reaches $\sim1.4$ near the disc plane. The integrated $<μ_b>$ pattern signals a superposition of bar rotation and internal streaming motion, with the near part shrinking in latitude and the far part expanding. To understand and interpret these remarkable data, we compare to a made-to-measure barred dynamical model, folding in the VIRAC selection function to construct mock maps. We find that our model of the barred bulge, with a pattern speed of 37.5 $\mathrm{km \, s^{-1} \, kpc^{-1}}$, is able to reproduce all observed features impressively well. Dynamical models like this will be key to unlocking the full potential of these data.
△ Less
Submitted 5 December, 2019; v1 submitted 5 March, 2019;
originally announced March 2019.
