-
Stabilizing Extreme Q-learning by Maclaurin Expansion
Authors:
Motoki Omura,
Takayuki Osa,
Yusuke Mukuta,
Tatsuya Harada
Abstract:
In Extreme Q-learning (XQL), Gumbel Regression is performed with an assumed Gumbel distribution for the error distribution. This allows learning of the value function without sampling out-of-distribution actions and has shown excellent performance mainly in Offline RL. However, issues remained, including the exponential term in the loss function causing instability and the potential for an error d…
▽ More
In Extreme Q-learning (XQL), Gumbel Regression is performed with an assumed Gumbel distribution for the error distribution. This allows learning of the value function without sampling out-of-distribution actions and has shown excellent performance mainly in Offline RL. However, issues remained, including the exponential term in the loss function causing instability and the potential for an error distribution diverging from the Gumbel distribution. Therefore, we propose Maclaurin Expanded Extreme Q-learning to enhance stability. In this method, applying Maclaurin expansion to the loss function in XQL enhances stability against large errors. It also allows adjusting the error distribution assumption from normal to Gumbel based on the expansion order. Our method significantly stabilizes learning in Online RL tasks from DM Control, where XQL was previously unstable. Additionally, it improves performance in several Offline RL tasks from D4RL, where XQL already showed excellent results.
△ Less
Submitted 7 June, 2024;
originally announced June 2024.
-
Internal 1000 AU-scale Structures of the R CrA Cluster-forming Cloud -- I: Filamentary Structures
Authors:
Kengo Tachihara,
Naofumi Fukaya,
Kazuki Tokuda,
Yasumasa Yamasaki,
Takeru Nishioka,
Daisei Abe,
Tsuyoshi Inoue,
Naoto Harada,
Ayumu Shoshi,
Shingo Nozaki,
Asako Sato,
Mitsuki Omura,
Kakeru Fujishiro,
Misato Fukagawa,
Masahiro N. Machida,
Takahiro Kanai,
Yumiko Oasa,
Toshikazu Onishi,
Kazuya Saigo,
Yasuo Fukui
Abstract:
We report on ALMA ACA observations of a high-density region of the Corona Australis cloud forming a young star cluster, and the results of resolving internal structures. In addition to embedded Class 0/I protostars in continuum, a number of complex dense filamentary structures are detected in the C18O and SO lines by the 7m array. These are sub-structures of the molecular clump that are detected b…
▽ More
We report on ALMA ACA observations of a high-density region of the Corona Australis cloud forming a young star cluster, and the results of resolving internal structures. In addition to embedded Class 0/I protostars in continuum, a number of complex dense filamentary structures are detected in the C18O and SO lines by the 7m array. These are sub-structures of the molecular clump that are detected by the TP array as the extended emission. We identify 101 and 37 filamentary structures with a few thousand AU widths in C18O and SO, respectively, called as feathers. The typical column density of the feathers in C18O is about 10^{22} cm^{-2}, and the volume density and line mass are ~ 10^5 cm^{-3}, and a few times M_{sun} pc^{-1}, respectively. This line mass is significantly smaller than the critical line mass expected for cold and dense gas. These structures have complex velocity fields, indicating a turbulent internal property. The number of feathers associated with Class 0/I protostars is only ~ 10, indicating that most of them do not form stars but rather being transient structures. The formation of feathers can be interpreted as a result of colliding gas flow as the morphology well reproduced by MHD simulations, supported by the the presence of HI shells in the vicinity. The colliding gas flows may accumulate gas and form filaments and feathers, and trigger the active star formation of the R CrA cluster.
△ Less
Submitted 17 April, 2024;
originally announced April 2024.
-
Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning
Authors:
Motoki Omura,
Takayuki Osa,
Yusuke Mukuta,
Tatsuya Harada
Abstract:
In deep reinforcement learning, estimating the value function to evaluate the quality of states and actions is essential. The value function is often trained using the least squares method, which implicitly assumes a Gaussian error distribution. However, a recent study suggested that the error distribution for training the value function is often skewed because of the properties of the Bellman ope…
▽ More
In deep reinforcement learning, estimating the value function to evaluate the quality of states and actions is essential. The value function is often trained using the least squares method, which implicitly assumes a Gaussian error distribution. However, a recent study suggested that the error distribution for training the value function is often skewed because of the properties of the Bellman operator, and violates the implicit assumption of normal error distribution in the least squares method. To address this, we proposed a method called Symmetric Q-learning, in which the synthetic noise generated from a zero-mean distribution is added to the target values to generate a Gaussian error distribution. We evaluated the proposed method on continuous control benchmark tasks in MuJoCo. It improved the sample efficiency of a state-of-the-art reinforcement learning method by reducing the skewness of the error distribution.
△ Less
Submitted 12 March, 2024;
originally announced March 2024.
-
Discovery of Asymmetric Spike-like Structures of the 10 au Disk around the Very Low-luminosity Protostar Embedded in the Taurus Dense Core MC 27/L1521F with ALMA
Authors:
Kazuki Tokuda,
Naoto Harada,
Mitsuki Omura,
Tomoaki Matsumoto,
Toshikazu Onishi,
Kazuya Saigo,
Ayumu Shoshi,
Shingo Nozaki,
Kengo Tachihara,
Naofumi Fukaya,
Yasuo Fukui,
Shu-ichiro Inutsuka,
Masahiro N. Machida
Abstract:
Recent Atacama Large Millimeter/submillimeter Array (ALMA) observations have revealed an increasing number of compact protostellar disks with radii of less than a few tens of astronomical units and that young Class 0/I objects have an intrinsic size diversity. To deepen our understanding of the origin of such tiny disks, we performed the highest-resolution configuration observations with ALMA at a…
▽ More
Recent Atacama Large Millimeter/submillimeter Array (ALMA) observations have revealed an increasing number of compact protostellar disks with radii of less than a few tens of astronomical units and that young Class 0/I objects have an intrinsic size diversity. To deepen our understanding of the origin of such tiny disks, we performed the highest-resolution configuration observations with ALMA at a beam size of $\sim$0$''$03 (4 au) on the very low-luminosity Class 0 protostar embedded in the Taurus dense core MC 27/L1521F. The 1.3 mm continuum measurement successfully resolved a tiny, faint ($\sim$1 mJy) disk with a major axis length of $\sim$10 au, one of the smallest examples in the ALMA protostellar studies. In addition, we detected spike-like components in the northeastern direction at the disk edge. Gravitational instability or other fragmentation mechanisms cannot explain the structures, given the central stellar mass of $\sim$0.2 $M_{\odot}$ and the disk mass of $\gtrsim$10$^{-4}$ $M_{\odot}$. Instead, we propose that these small spike structures were formed by a recent dynamic magnetic flux transport event due to interchange instability that would be favorable to occur if the parental core has a strong magnetic field. The presence of complex arc-like structures on a larger ($\sim$2000 au) scale in the same direction as the spike structures suggests that the event was not single. Such episodic, dynamical events may play an important role in maintaining the compact nature of the protostellar disk in the complex gas envelope during the main accretion phase.
△ Less
Submitted 3 April, 2024; v1 submitted 1 March, 2024;
originally announced March 2024.
-
An Extremely Young Protostellar Core, MMS 1/ OMC-3: Episodic Mass Ejection History Traced by the Micro SiO Jet
Authors:
Satoko Takahashi,
Masahiro N. Machida,
Mitsuki Omura,
Doug Johnstone,
Kazuya Saigo,
Naoto Harada,
Kohji Tomisaka,
Paul T. P. Ho,
Luis A. Zapata,
Steve Mairs,
Gregory J. Herczeg,
Kotomi Taniguchi,
Yuhua Liu,
Asako Sato
Abstract:
We present ${\sim}0.2$ arcsec ($\sim$80 au) resolution observations of the CO (2-1) and SiO (5-4) lines made with the Atacama large millimeter/submillimeter array toward an extremely young intermediate-mass protostellar source (t$_{\rm dyn}<$1000 years), MMS 1 located in the Orion Molecular Cloud-3 region. We have successfully imaged a very compact CO molecular outflow associated with MMS 1, havin…
▽ More
We present ${\sim}0.2$ arcsec ($\sim$80 au) resolution observations of the CO (2-1) and SiO (5-4) lines made with the Atacama large millimeter/submillimeter array toward an extremely young intermediate-mass protostellar source (t$_{\rm dyn}<$1000 years), MMS 1 located in the Orion Molecular Cloud-3 region. We have successfully imaged a very compact CO molecular outflow associated with MMS 1, having deprojected lobe sizes of $\sim$18000 au (red-shifted lobe) and $\sim$35000 au (blue-shifted lobe). We have also detected an extremely compact ($\lesssim$1000 au) and collimated SiO protostellar jet within the CO outflow. The maximum deprojected jet speed is measured to be as high as 93 km s$^{-1}$. The SiO jet wiggles and displays a chain of knots. Our detection of the molecular outflow and jet is the first direct evidence that MMS 1 already hosts a protostar. The position-velocity diagram obtained from the SiO emission shows two distinct structures: (i) bow-shocks associated with the tips of the outflow, and (ii) a collimated jet, showing the jet velocities linearly increasing with the distance from the driving source. Comparisons between the observations and numerical simulations quantitatively share similarities such as multiple-mass ejection events within the jet and Hubble-like flow associated with each mass ejection event. Finally, while there is a weak flux decline seen in the 850 $μ$m light curve obtained with JCMT/SCUBA 2 toward MMS 1, no dramatic flux change events are detected. This suggests that there has not been a clear burst event within the last 8 years.
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
Revealing multiple nested molecular outflows with rotating signatures in HH270mms1-A with ALMA
Authors:
Mitsuki Omura,
Kazuki Tokuda,
Masahiro N. Machida
Abstract:
We present molecular line observations of the protostellar outflow associated with HH270mms1 in the Orion B molecular cloud with ALMA. The 12CO(J = 3 - 2) emissions show that the outflow velocity structure consists of four distinct components of low ($\gtrsim$ 10 km s-1), intermediate (~ 10 - 25 km s-1) and high ($\gtrsim$ 40 km s-1) velocities in addition to the entrained gas velocity (~ 25 - 40…
▽ More
We present molecular line observations of the protostellar outflow associated with HH270mms1 in the Orion B molecular cloud with ALMA. The 12CO(J = 3 - 2) emissions show that the outflow velocity structure consists of four distinct components of low ($\gtrsim$ 10 km s-1), intermediate (~ 10 - 25 km s-1) and high ($\gtrsim$ 40 km s-1) velocities in addition to the entrained gas velocity (~ 25 - 40 km s-1). The high- and intermediate-velocity flows have well-collimated structures surrounded by the low-velocity flow. The chain of knots is embedded in the high-velocity flow or jet, which is the evidence of episodic mass ejections induced by time-variable mass accretion. We could detect the velocity gradients perpendicular to the outflow axis in both the low- and intermediate-velocity flows. We confirmed the rotation of the envelope and disk in the 13CO and C17O emission and found that their velocity gradients are the same as those of the outflow. Thus, we concluded that the velocity gradients in the low- and intermediate-velocity flows are due to the outflow rotation. Using observational outflow properties, we estimated the outflow launching radii to be 67.1 - 77.1 au for the low-velocity flow and 13.3 - 20.8 au for the intermediate-velocity flow. Although we could not detect the rotation in the jets due to the limited spatial resolution, we estimated the jet launching radii to be (2.36 - 3.14) x 10^-2 au using the observed velocity of each knots. Thus, the jet is driven from the inner disk region. We could identify the launching radii of distinct velocity components within a single outflow with all the prototypical characteristics expected from recent theoretical works.
△ Less
Submitted 5 January, 2024;
originally announced January 2024.
-
Ring Gap Structure around Class I Protostar WL 17
Authors:
Ayumu Shoshi,
Naoto Harada,
Kazuki Tokuda,
Yoshihiro Kawasaki,
Hayao Yamasaki,
Asako Sato,
Mitsuki Omura,
Masayuki Yamaguchi,
Kengo Tachihara,
Masahiro N. Machida
Abstract:
WL 17 is a Class I object and was considered to have a ring-hole structure. We analyzed the structure around WL 17 to investigate the detailed properties of WL 17. We used ALMA archival data, which have a higher angular resolution than previous observations. We investigated the WL 17 system with the 1.3 mm dust continuum and 12CO and C18O (J = 2-1) line emissions. The dust continuum emission showe…
▽ More
WL 17 is a Class I object and was considered to have a ring-hole structure. We analyzed the structure around WL 17 to investigate the detailed properties of WL 17. We used ALMA archival data, which have a higher angular resolution than previous observations. We investigated the WL 17 system with the 1.3 mm dust continuum and 12CO and C18O (J = 2-1) line emissions. The dust continuum emission showed a clear ring structure with inner and outer edges of ~11 and ~21 au, respectively. In addition, we detected an inner disk of < 5 au radius enclosing the central star within the ring, the first observation of this structure. Thus, WL 17 has a ring-gap structure, not a ring-hole structure. We did not detect any marked emission in either the gap or inner disk, indicating that there is no sign of a planet, circumplanetary disk, or binary companion. We identified the base of both blue-shifted and red-shifted outflows based on the 12CO emission, which is clearly associated with the disk around WL 17. The outflow mass ejection rate is ~3.6x10^-7 Msun yr-1 and the dynamical timescale is as short as ~ 10^4 yr. The C18O emission showed that an inhomogeneous infalling envelope, which can induce episodic mass accretion, is distributed in the region within ~1000 au from the central protostar. With these new findings, we can constrain the planet formation and dust growth scenarios in the accretion phase of star formation.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
An ALMA-resolved view of 7000 au Protostellar Gas Ring around the Class I source CrA-IRS 2 as a possible sign of magnetic flux advection
Authors:
Kazuki Tokuda,
Naofumi Fukaya,
Kengo Tachihara,
Mitsuki Omura,
Naoto Harada,
Shingo Nozaki,
Ayumu Shoshi,
Masahiro N. Machida
Abstract:
Transferring a significant fraction of the magnetic flux from a dense cloud core is essential in the star formation process. A ring-like structure produced by magnetic flux loss has been predicted theoretically, but no observational identification has been presented. We have performed ALMA observations of the Class I protostar IRS 2 in the Corona Australis star-forming region and resolved a distin…
▽ More
Transferring a significant fraction of the magnetic flux from a dense cloud core is essential in the star formation process. A ring-like structure produced by magnetic flux loss has been predicted theoretically, but no observational identification has been presented. We have performed ALMA observations of the Class I protostar IRS 2 in the Corona Australis star-forming region and resolved a distinctive gas ring in the C$^{18}$O ($J$ = 2-1) line emission. The center of this gas ring is $\sim$5,000 au away from the protostar, with a diameter of $\sim$7,000 au. The radial velocity of the gas is $\lesssim1$ km s$^{-1}$ blueshifted from that of the protostar, with a possible expanding feature judged from the velocity-field (moment 1) map and position-velocity diagram. These features are either observationally new or have been discovered but not discussed in depth because they are difficult to explain by well-studied protostellar phenomena such as molecular outflows and accretion streamers. A plausible interpretation is a magnetic wall created by the advection of magnetic flux which is theoretically expected in the Class 0/I phase during star formation as a removal mechanism of magnetic flux. Similar structures reported in the other young stellar sources could likely be candidates formed by the same mechanism, encouraging us to revisit the issue of magnetic flux transport in the early stages of star formation from an observational perspective.
△ Less
Submitted 15 October, 2023; v1 submitted 24 September, 2023;
originally announced September 2023.
-
Crescent-Shaped Molecular Outflow from the Intermediate-mass Protostar DK Cha Revealed by ALMA
Authors:
Naoto Harada,
Kazuki Tokuda,
Hayao Yamasaki,
Asako Sato,
Mitsuki Omura,
Shingo Hirano,
Toshikazu Onishi,
Kengo Tachihara,
Masahiro N. Machida
Abstract:
We report on an Atacama Large Millimeter/submillimeter Array (ALMA) study of the Class I or II intermediate-mass protostar DK Cha in the Chamaeleon II region. The 12CO (J=2-1) images have an angular resolution of ~1'' (~250 au) and show high-velocity blueshifted (>70 km s-1) and redshifted (>50 km s-1) emissions which have 3000 au scale crescent-shaped structures around the protostellar disk trace…
▽ More
We report on an Atacama Large Millimeter/submillimeter Array (ALMA) study of the Class I or II intermediate-mass protostar DK Cha in the Chamaeleon II region. The 12CO (J=2-1) images have an angular resolution of ~1'' (~250 au) and show high-velocity blueshifted (>70 km s-1) and redshifted (>50 km s-1) emissions which have 3000 au scale crescent-shaped structures around the protostellar disk traced in the 1.3mm continuum. Because the high-velocity components of the CO emission are associated with the protostar, we concluded that the emission traces the pole-on outflow. The blueshifted outflow lobe has a clear layered velocity gradient with a higher velocity component located on the inner side of the crescent shape, which can be explained by a model of an outflow with a higher velocity in the inner radii. Based on the directly driven outflow scenario, we estimated the driving radii from the observed outflow velocities and found that the driving region extends over two orders of magnitude. The 13CO emission traces a complex envelope structure with arc-like substructures with lengths of ~1000au. We identified the arc-like structures as streamers because they appear to be connected to a rotating infalling envelope. DK Cha is useful for understanding characteristics that are visible by looking at nearly face-on configurations of young protostellar systems, providing an alternative perspective for studying the star-formation process.
△ Less
Submitted 3 February, 2023;
originally announced February 2023.