-
Optimizing the Long-Term Behaviour of Deep Reinforcement Learning for Pushing and Gras**
Authors:
Rodrigo Chau
Abstract:
We investigate the "Visual Pushing for Gras**" (VPG) system by Zeng et al. and the "Hourglass" system by Ewerton et al., an evolution of the former. The focus of our work is the investigation of the capabilities of both systems to learn long-term rewards and policies. Zeng et al. original task only needs a limited amount of foresight. Ewerton et al. attain their best performance using an agent w…
▽ More
We investigate the "Visual Pushing for Gras**" (VPG) system by Zeng et al. and the "Hourglass" system by Ewerton et al., an evolution of the former. The focus of our work is the investigation of the capabilities of both systems to learn long-term rewards and policies. Zeng et al. original task only needs a limited amount of foresight. Ewerton et al. attain their best performance using an agent which only takes the most immediate action under consideration. We are interested in the ability of their models and training algorithms to accurately predict long-term Q-Values. To evaluate this ability, we design a new bin sorting task and reward function. Our task requires agents to accurately estimate future rewards and therefore use high discount factors in their Q-Value calculation. We investigate the behaviour of an adaptation of the VPG training algorithm on our task. We show that this adaptation can not accurately predict the required long-term action sequences. In addition to the limitations identified by Ewerton et al., it suffers from the known Deep Q-Learning problem of overestimated Q-Values. In an effort to solve our task, we turn to the Hourglass models and combine them with the Double Q-Learning approach. We show that this approach enables the models to accurately predict long-term action sequences when trained with large discount factors. Our results show that the Double Q-Learning technique is essential for training with very high discount factors, as the models Q-Value predictions diverge otherwise. We also experiment with different approaches for discount factor scheduling, loss calculation and exploration procedures. Our results show that the latter factors do not visibly influence the model's performance for our task.
△ Less
Submitted 7 April, 2022;
originally announced April 2022.
-
ConVIScope: Visual Analytics for Exploring Patient Conversations
Authors:
Raymond Li,
Enamul Hoque,
Giuseppe Carenini,
Richard Lester,
Raymond Chau
Abstract:
The proliferation of text messaging for mobile health is generating a large amount of patient-doctor conversations that can be extremely valuable to health care professionals. We present ConVIScope, a visual text analytic system that tightly integrates interactive visualization with natural language processing in analyzing patient-doctor conversations. ConVIScope was developed in collaboration wit…
▽ More
The proliferation of text messaging for mobile health is generating a large amount of patient-doctor conversations that can be extremely valuable to health care professionals. We present ConVIScope, a visual text analytic system that tightly integrates interactive visualization with natural language processing in analyzing patient-doctor conversations. ConVIScope was developed in collaboration with healthcare professionals following a user-centered iterative design. Case studies with six domain experts suggest the potential utility of ConVIScope and reveal lessons for further developments.
△ Less
Submitted 30 August, 2021;
originally announced August 2021.
-
Susceptibility inhomogeneity and non-Fermi liquid behavior in UCu_{5-x}Pt_x
Authors:
D. E. MacLaughlin,
M. S. Rose,
O. O. Bernal,
R. H. Heffner,
G. J. Nieuwenhuys,
R. Chau,
M. B. Maple
Abstract:
Transverse-field muSR shifts and relaxation rates have been measured in the non-Fermi liquid (NFL) alloy system UCu_{5-x}Pt_x, x = 1.0, 1.5, and 2.5. At low temperatures the fractional spread in Knight shifts delta K/K approx deltachi/chi is gtrsim 2 for x = 1, but is only half this value for x = 1.5 and 2.5. In a disorder-driven scenario where the NFL behavior is due to a broadly distributed (K…
▽ More
Transverse-field muSR shifts and relaxation rates have been measured in the non-Fermi liquid (NFL) alloy system UCu_{5-x}Pt_x, x = 1.0, 1.5, and 2.5. At low temperatures the fractional spread in Knight shifts delta K/K approx deltachi/chi is gtrsim 2 for x = 1, but is only half this value for x = 1.5 and 2.5. In a disorder-driven scenario where the NFL behavior is due to a broadly distributed (Kondo or Griffiths-phase cluster) characteristic energy E, our results indicate that delta E/E_{rm av} approx (delta K/K)_{T=0} is similar for UCu_{5-x}Pd_x (x = 1 and 1.5) and UCu_4Pt, but is reduced for UCu_{5-x}Pt_x, x = 1.5 and 2.5. This reduction is due to a marked increase of E with increasing x; the spread delta E is found to be roughly independent of x. Our results correlate with the observed suppression of other NFL anomalies for x > 1 in UCu_{5-x}Pt_x but not in UCu_{5-x}Pd_x, and are further evidence for the importance of disorder in the NFL behavior of both these alloy systems.
△ Less
Submitted 30 August, 2005;
originally announced August 2005.
-
Effects of lattice disorder in the UCu_(5-x)Pd_(x) system
Authors:
E. D. Bauer,
C. H. Booth,
G. H. Kwei,
R. Chau,
M. B. Maple
Abstract:
The UCu(5-x)Pd(x) system exhibits non-Fermi liquid (NFL) in the thermodynamic and transport properties at low temperatures for Pd concentrations 0.9 <~ x <~ 1.5. The local structure around the U, Cu, and Pd atoms has been measured for 0 <= x <= 1.5 using the X-ray Absorption Fine Structure (XAFS) technique in order to quantify the effects of lattice disorder on the NFL properties. A model which…
▽ More
The UCu(5-x)Pd(x) system exhibits non-Fermi liquid (NFL) in the thermodynamic and transport properties at low temperatures for Pd concentrations 0.9 <~ x <~ 1.5. The local structure around the U, Cu, and Pd atoms has been measured for 0 <= x <= 1.5 using the X-ray Absorption Fine Structure (XAFS) technique in order to quantify the effects of lattice disorder on the NFL properties. A model which allows a percentage of the Pd atoms to occupy nominal Cu (16e) sites, s, was used to fit the Pd and Cu K edge and U L(III) edge data. Pd/Cu site interchange was found to occur in all samples (x != 0), reaching a minimum value of s ~0.17 at x = 0.7 and increasing monotonically to s ~ 0.4 at x=1.5. These data also determine the static disorder the nearest neighbor U--Cu pairs. The results indicate that the measured U--Cu static disorder is not sufficient to explain the NFL behavior of the magnetic susceptibility within the single-ion Kondo disorder model and casts doubt on the applicability of this model to UCu(5-x)Pd(x).
△ Less
Submitted 7 May, 2002;
originally announced May 2002.
-
Slow Spin Dynamics in Non-Fermi-Liquid UCu_{5-x}Pd_x, x = 1.0 and 1.5
Authors:
D. E. MacLaughlin,
R. H. Heffner,
O. O. Bernal,
G. J. Nieuwenhuys,
J. E. Sonier,
M. S. Rose,
R. Chau,
M. B. Maple,
B. Andraka
Abstract:
Low-temperature muon spin-lattice relaxation measurements in the non-Fermi-liquid heavy-fermion alloys UCu_{5-x}Pd_x, x = 1.0 and 1.5, indicate inhomogeneously distributed f-electron spin fluctuation rates, and exhibit a time-field scaling of the muon relaxation function indicative of long-lived spin correlations. In UCu_4Pd the scaling exponent γis small and temperature independent. In UCu_{3.5…
▽ More
Low-temperature muon spin-lattice relaxation measurements in the non-Fermi-liquid heavy-fermion alloys UCu_{5-x}Pd_x, x = 1.0 and 1.5, indicate inhomogeneously distributed f-electron spin fluctuation rates, and exhibit a time-field scaling of the muon relaxation function indicative of long-lived spin correlations. In UCu_4Pd the scaling exponent γis small and temperature independent. In UCu_{3.5}Pd_{1.5} γvaries with temperature, increasing with decreasing temperature similar to spin-glass AgMn. This suggests that the spin-glass state found for x \gtrsim 2 in UCu_{5-x}Pd_x modifies the low-frequency spin dynamics in UCu_{3.5}Pd_{1.5}.
△ Less
Submitted 4 September, 2001;
originally announced September 2001.
-
Glassy Spin Dynamics in Non-Fermi-Liquid UCu_{5-x}Pd_x, x = 1.0 and 1.5
Authors:
D. E. MacLaughlin,
O. O. Bernal,
R. H. Heffner,
G. J. Nieuwenhuys,
M. S. Rose,
J. E. Sonier,
B. Andraka,
R. Chau,
M. B. Maple
Abstract:
Local f-electron spin dynamics in the non-Fermi-liquid heavy-fermion alloys UCu_{5-x}Pd_x, x = 1.0 and 1.5, have been studied using muon spin-lattice relaxation. The sample-averaged asymmetry function Gbar(t) indicates strongly inhomogeneous spin fluctuations, and exhibits the scaling Gbar(t,H) = Gbar(t/H^γ) expected from glassy dynamics. At 0.05 K γ(x=1.0) = 0.35 \pm 0.1, but γ(x=1.5) = 0.7 \pm…
▽ More
Local f-electron spin dynamics in the non-Fermi-liquid heavy-fermion alloys UCu_{5-x}Pd_x, x = 1.0 and 1.5, have been studied using muon spin-lattice relaxation. The sample-averaged asymmetry function Gbar(t) indicates strongly inhomogeneous spin fluctuations, and exhibits the scaling Gbar(t,H) = Gbar(t/H^γ) expected from glassy dynamics. At 0.05 K γ(x=1.0) = 0.35 \pm 0.1, but γ(x=1.5) = 0.7 \pm 0.1. This is in contrast to inelastic neutron scattering results, which yield γ= 0.33 for both concentrations. There is no sign of static magnetism \gtrsim 10^{-3} μ_B/U ion in either material above 0.05 K. Our results strongy suggest that both alloys are quantum spin glasses.
△ Less
Submitted 19 June, 2001; v1 submitted 20 December, 2000;
originally announced December 2000.
-
Pd/Cu Site Interchange and Non-Fermi-Liquid Behavior in UCu_4Pd
Authors:
C. H. Booth,
D. E. MacLaughlin,
R. H. Heffner,
R. Chau,
M. B Maple,
G. H. Kwei
Abstract:
X-ray-absorption fine-structure measurements of the local structure in UCu_4Pd are described which indicate a probable lattice-disorder origin for non-Fermi-liquid behavior in this material. Short Pd-Cu distances are observed, consistent with 24 +/- 3% of the Pd atoms occupying nominally Cu sites. A "Kondo disorder" model, based on the effect on the local Kondo temperature T_K of this interchang…
▽ More
X-ray-absorption fine-structure measurements of the local structure in UCu_4Pd are described which indicate a probable lattice-disorder origin for non-Fermi-liquid behavior in this material. Short Pd-Cu distances are observed, consistent with 24 +/- 3% of the Pd atoms occupying nominally Cu sites. A "Kondo disorder" model, based on the effect on the local Kondo temperature T_K of this interchange and some additional bond-length disorder, agrees quantitatively with previous experimental susceptibility data, and therefore also with specific heat and magnetic resonance experiments.
△ Less
Submitted 23 September, 1998;
originally announced September 1998.
-
Susceptibility inhomogeneity and non-Fermi liquid behavior in nominally ordered UCu_4Pd
Authors:
D. E. MacLaughlin,
R. H. Heffner,
G. J. Nieuwenhuys,
G. M. Luke,
Y. Fudamoto,
Y. J. Uemura,
R. Chau,
M. B. Maple,
B. Andraka
Abstract:
Muon spin rotation experiments on a stoichiometric sample of the non-Fermi liquid (NFL) heavy-fermion compound UCu_4Pd, in which recent neutron Bragg scattering measurements are consistent with an ordered structure, indicate that the U-ion susceptibility is strongly inhomogeneous at low temperatures. This suggests that residual disorder dominates NFL behavior. The data also indicate a short corr…
▽ More
Muon spin rotation experiments on a stoichiometric sample of the non-Fermi liquid (NFL) heavy-fermion compound UCu_4Pd, in which recent neutron Bragg scattering measurements are consistent with an ordered structure, indicate that the U-ion susceptibility is strongly inhomogeneous at low temperatures. This suggests that residual disorder dominates NFL behavior. The data also indicate a short correlation length (\lesssim 1 lattice spacing) for this inhomogeneity and a rapid low-temperature U-moment relaxation rate (\gtrsim 10^{12} s^{-1}), which constrain cluster-based models of NFL behavior.
△ Less
Submitted 17 September, 1998; v1 submitted 30 April, 1998;
originally announced April 1998.
-
Evidence for a common physical description of non-Fermi-liquid behavior in f-electron systems
Authors:
M. C. de Andrade,
R. Chau,
R. P. Dickey,
N. R. Dilley,
E. J. Freeman,
D. A. Gajewski,
M. B. Maple,
R. Movshovich,
A. H. Castro Neto,
G. E. Castilla,
B. A. Jones
Abstract:
The non-Fermi-liquid (NFL) behavior observed in the low temperature specific heat $C(T)$ and magnetic susceptibility $χ(T)$ of f-electron systems is analyzed within the context of a recently developed theory based on Griffiths singularities. Measurements of $C(T)$ and $χ(T)$ in the systems $Th_{1-x}U_{x}Pd_{2}Al_{3}$, $Y_{1-x}U_{x}Pd_3$, and $UCu_{5-x}M_{x}$ (M = Pd, Pt) are found to be consiste…
▽ More
The non-Fermi-liquid (NFL) behavior observed in the low temperature specific heat $C(T)$ and magnetic susceptibility $χ(T)$ of f-electron systems is analyzed within the context of a recently developed theory based on Griffiths singularities. Measurements of $C(T)$ and $χ(T)$ in the systems $Th_{1-x}U_{x}Pd_{2}Al_{3}$, $Y_{1-x}U_{x}Pd_3$, and $UCu_{5-x}M_{x}$ (M = Pd, Pt) are found to be consistent with $C(T)/T \propto χ(T) \propto T^{-1+λ}$ predicted by this model with $λ<1$ in the NFL regime. These results suggest that the NFL properties observed in a wide variety of f-electron systems can be described within the context of a common physical picture.
△ Less
Submitted 7 February, 1998;
originally announced February 1998.
