-
Software engineering to sustain a high-performance computing scientific application: QMCPACK
Authors:
William F. Godoy,
Steven E. Hahn,
Michael M. Walsh,
Philip W. Fackler,
Jaron T. Krogel,
Peter W. Doak,
Paul R. C. Kent,
Alfredo A. Correa,
Ye Luo,
Mark Dewing
Abstract:
We provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum Monte Carlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion of continuous integration (CI) targeting CPUs, using GitHub Actions runners, and NVIDIA and AMD GPUs in pre-exascale systems, using self-hosted hard…
▽ More
We provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum Monte Carlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion of continuous integration (CI) targeting CPUs, using GitHub Actions runners, and NVIDIA and AMD GPUs in pre-exascale systems, using self-hosted hardware; (ii) incremental reduction of memory leaks using sanitizers, (iii) incorporation of Docker containers for CI and reproducibility, and (iv) refactoring efforts to improve maintainability, testing coverage, and memory lifetime management. We quantify the value of these improvements by providing metrics to illustrate the shift towards a predictive, rather than reactive, sustainable maintenance approach. Our goal, in documenting the impact of these efforts on QMCPACK, is to contribute to the body of knowledge on the importance of research software engineering (RSE) for the sustainability of community HPC codes and scientific discovery at scale.
△ Less
Submitted 21 July, 2023;
originally announced July 2023.
-
Efficient loading of reduced data ensembles produced at ORNL SNS/HFIR neutron time-of-flight facilities
Authors:
William F Godoy,
Andrei T Savici,
Steven E Hahn,
Peter F Peterson
Abstract:
We present algorithmic improvements to the loading operations of certain reduced data ensembles produced from neutron scattering experiments at Oak Ridge National Laboratory (ORNL) facilities. Ensembles from multiple measurements are required to cover a wide range of the phase space of a sample material of interest. They are stored using the standard NeXus schema on individual HDF5 files. This mak…
▽ More
We present algorithmic improvements to the loading operations of certain reduced data ensembles produced from neutron scattering experiments at Oak Ridge National Laboratory (ORNL) facilities. Ensembles from multiple measurements are required to cover a wide range of the phase space of a sample material of interest. They are stored using the standard NeXus schema on individual HDF5 files. This makes it a scalability challenge, as the number of experiments stored increases in a single ensemble file. The present work follows up on our previous efforts on data management algorithms, to address identified input output (I/O) bottlenecks in Mantid, an open-source data analysis framework used across several neutron science facilities around the world. We reuse an in-memory binary-tree metadata index that resembles data access patterns, to provide a scalable search and extraction mechanism. In addition, several memory operations are refactored and optimized for the current common use cases, ranging most frequently from 10 to 180, and up to 360 separate measurement configurations. Results from this work show consistent speed ups in wall-clock time on the Mantid LoadMD routine, ranging from 19\% to 23\% on average, on ORNL production computing systems. The latter depends on the complexity of the targeted instrument-specific data and the system I/O and compute variability for the shared computational resources available to users of ORNL's Spallation Neutron Source (SNS) and the High Flux Isotope Reactor (HFIR) instruments. Nevertheless, we continue to highlight the need for more research to address reduction challenges as experimental data volumes, user time and processing costs increase.
△ Less
Submitted 30 November, 2021;
originally announced December 2021.
-
Efficient Data Management in Neutron Scattering Data Reduction Workflows at ORNL
Authors:
William F Godoy,
Peter F Peterson,
Steven E Hahn,
Jay J Billings
Abstract:
Oak Ridge National Laboratory (ORNL) experimental neutron science facilities produce 1.2\,TB a day of raw event-based data that is stored using the standard metadata-rich NeXus schema built on top of the HDF5 file format. Performance of several data reduction workflows is largely determined by the amount of time spent on the loading and processing algorithms in Mantid, an open-source data analysis…
▽ More
Oak Ridge National Laboratory (ORNL) experimental neutron science facilities produce 1.2\,TB a day of raw event-based data that is stored using the standard metadata-rich NeXus schema built on top of the HDF5 file format. Performance of several data reduction workflows is largely determined by the amount of time spent on the loading and processing algorithms in Mantid, an open-source data analysis framework used across several neutron sciences facilities around the world. The present work introduces new data management algorithms to address identified input output (I/O) bottlenecks on Mantid. First, we introduce an in-memory binary-tree metadata index that resemble NeXus data access patterns to provide a scalable search and extraction mechanism. Second, data encapsulation in Mantid algorithms is optimally redesigned to reduce the total compute and memory runtime footprint associated with metadata I/O reconstruction tasks. Results from this work show speed ups in wall-clock time on ORNL data reduction workflows, ranging from 11\% to 30\% depending on the complexity of the targeted instrument-specific data. Nevertheless, we highlight the need for more research to address reduction challenges as experimental data volumes increase.
△ Less
Submitted 5 January, 2021;
originally announced January 2021.
-
Magnetic Frustration Driven by Itinerancy in Spinel CoV2O4
Authors:
J. H. Lee,
J. Ma,
S. E. Hahn,
H. B. Cao,
Tao Hong,
M. S. Yeom,
S. Okamoto,
H. D. Zhou,
M. Matsuda,
R. S. Fishman
Abstract:
Localized spins and itinerant electrons rarely coexist in geometrically-frustrated spinel lattices. We show that the spinel CoV2O4 stands at the crossover from insulating to itinerant behavior and exhibits a complex interplay between localized spins and itinerant electrons. In contrast to the expected paramagnetism, localized spins supported by enhanced exchange couplings are frustrated by the eff…
▽ More
Localized spins and itinerant electrons rarely coexist in geometrically-frustrated spinel lattices. We show that the spinel CoV2O4 stands at the crossover from insulating to itinerant behavior and exhibits a complex interplay between localized spins and itinerant electrons. In contrast to the expected paramagnetism, localized spins supported by enhanced exchange couplings are frustrated by the effects of delocalized electrons. This frustration produces a non-collinear spin state and may be responsible for macroscopic spin-glass behavior. Competing phases can be uncovered by external perturbations such as pressure or magnetic field, which enhance the frustration.
△ Less
Submitted 9 February, 2017;
originally announced February 2017.
-
Extended Magnetic Exchange Interactions in the High-Temperature Ferromagnet MnBi
Authors:
T. J. Williams,
A. E. Taylor,
A. D. Christianson,
S. E. Hahn,
R. S. Fishman,
D. S. Parker,
M. A. McGuire,
B. C. Sales,
M. D. Lumsden
Abstract:
The high-temperature ferromagnet MnBi continues to receive attention as a candidate to replace rare-earth-containing permanent magnets in applications above room temperature. This is due to a high Curie temperature, large magnetic moments, and a coercivity that increases with temperature. The synthesis of MnBi also allows for crystals that are free of interstitial Mn, enabling more direct access t…
▽ More
The high-temperature ferromagnet MnBi continues to receive attention as a candidate to replace rare-earth-containing permanent magnets in applications above room temperature. This is due to a high Curie temperature, large magnetic moments, and a coercivity that increases with temperature. The synthesis of MnBi also allows for crystals that are free of interstitial Mn, enabling more direct access to the key interactions underlying the physical properties of binary Mn-based ferromagnets. In this work, we use inelastic neutron scattering to measure the spin waves of MnBi in order to characterize the magnetic exchange at low temperature. Consistent with the spin reorientation that occurs below 140 K, we do not observe a spin gap in this system above our experimental resolution. A Heisenberg model was fit to the spin wave data in order to characterize the long-range nature of the exchange. It was found that interactions up to sixth nearest neighbor are required to fully parameterize the spin waves. Surprisingly, the nearest-neighbor term is antiferromagnetic, and the realization of a ferromagnetic ground state relies on the more numerous ferromagnetic terms beyond nearest neighbor, suggesting that the ferromagnetic ground state arises as a consequence of the long-ranged interactions in the system.
△ Less
Submitted 10 March, 2016; v1 submitted 8 March, 2016;
originally announced March 2016.
-
Influence of interstitial Mn on magnetism in room-temperature ferromagnet Mn(1+delta)Sb
Authors:
A. E. Taylor,
T. Berlijn,
S. E. Hahn,
A. F. May,
T. J. Williams,
L. Poudel,
S. Calder,
R. S. Fishman,
M. B. Stone,
A. A. Aczel,
H. B. Cao,
M. D. Lumsden,
A. D. Christianson
Abstract:
We report elastic and inelastic neutron scattering measurements of the high-TC ferromagnet Mn(1+delta)Sb. Measurements were performed on a large, TC=434 K, single crystal with interstitial Mn content of delta~0.13. The neutron diffraction results reveal that the interstitial Mn has a magnetic moment, and that it is aligned antiparallel to the main Mn moment. We perform density functional theory ca…
▽ More
We report elastic and inelastic neutron scattering measurements of the high-TC ferromagnet Mn(1+delta)Sb. Measurements were performed on a large, TC=434 K, single crystal with interstitial Mn content of delta~0.13. The neutron diffraction results reveal that the interstitial Mn has a magnetic moment, and that it is aligned antiparallel to the main Mn moment. We perform density functional theory calculations including the interstitial Mn, and find the interstitial to be magnetic in agreement with the diffraction data. The inelastic neutron scattering measurements reveal two features in the magnetic dynamics: i) a spin-wave-like dispersion emanating from ferromagnetic Bragg positions (H K 2n), and ii) a broad, non-dispersive signal centered at forbidden Bragg positions (H$\,$K$\,$2$n$+1). The inelastic spectrum cannot be modeled by simple linear spin-wave theory calculations, and appears to be significantly altered by the presence of the interstitial Mn ions. The results show that the influence of the interstitial Mn on the magnetic state in this system is more important than previously understood.
△ Less
Submitted 12 June, 2015; v1 submitted 27 April, 2015;
originally announced April 2015.
-
Strong competition between orbital-ordering and itinerancy in a frustrated spinel vanadate
Authors:
J. Ma,
J. H. Lee,
S. E. Hahn,
Tao Hong,
H. B. Cao,
A. A. Aczel,
Z. L. Dun,
M. B. Stone,
W. Tian,
Y. Qiu,
J. R. D. Copley,
H. D. Zhou,
R. S. Fishman,
M. Matsuda
Abstract:
The crossover from localized- to itinerant-electron behavior is associated with many intriguing phenomena in condensed-matter physics. In this paper, we investigate the crossover from localized to itinerant regimes in the spinel system Mn$_{1-x}$Co$_x$V$_2$O$_4$. At low Co do**, orbital order (OO) of the localized electrons on the V3+ ions suppresses magnetic frustration by triggering a tetragon…
▽ More
The crossover from localized- to itinerant-electron behavior is associated with many intriguing phenomena in condensed-matter physics. In this paper, we investigate the crossover from localized to itinerant regimes in the spinel system Mn$_{1-x}$Co$_x$V$_2$O$_4$. At low Co do**, orbital order (OO) of the localized electrons on the V3+ ions suppresses magnetic frustration by triggering a tetragonal distortion. With Co do**, electronic itinerancy melts the OO and suppresses the structural phase transition while the reduced spin-lattice coupling produces magnetic frustration. Neutron scattering measurements and first-principles-guided spin models reveal that the non-collinear state at high Co do** is produced by weakened local anisotropy and enhanced Co-V spin interactions.
△ Less
Submitted 16 July, 2015; v1 submitted 15 July, 2014;
originally announced July 2014.
-
Inelastic neutron scattering studies of YFeO$_3$
Authors:
S. E. Hahn,
A. A. Podlesnyak,
R. S. Fishman,
G. Ehlers,
G. E. Granroth,
A. I. Kolesnikov,
E. Pomjakushina,
K. Conder
Abstract:
Spin waves in the the rare earth orthorferrite YFeO$_3$ have been studied by inelastic neutron scattering and analyzed with a full four-sublattice model including contributions from both the weak ferromagnetic and hidden antiferromagnetic orders. Antiferromagnetic (AFM) exchange interactions of $J_1 = -4.23 \pm 0.08$ (nearest-neighbors only) or $J_1 = -4.77 \pm 0.08$ meV and…
▽ More
Spin waves in the the rare earth orthorferrite YFeO$_3$ have been studied by inelastic neutron scattering and analyzed with a full four-sublattice model including contributions from both the weak ferromagnetic and hidden antiferromagnetic orders. Antiferromagnetic (AFM) exchange interactions of $J_1 = -4.23 \pm 0.08$ (nearest-neighbors only) or $J_1 = -4.77 \pm 0.08$ meV and $J_2 = -0.21 \pm 0.04$ meV lead to excellent fits for most branches at both low and high energies. An additional branch associated with the hidden antiferromagnetic order was observed. This work paves the way for studies of other materials in this class containing spin reorientation transitions and magnetic rare earth ions.
△ Less
Submitted 20 November, 2013; v1 submitted 14 September, 2013;
originally announced September 2013.
-
Magnetism dependent phonon anomaly in LaFeAsO observed via inelastic x-ray scattering
Authors:
S. E. Hahn,
G. S. Tucker,
J. -Q. Yan,
A. H. Said,
B. M. Leu,
R. W. McCallum,
E. E. Alp,
T. A. Lograsso,
R. J. McQueeney,
B. N. Harmon
Abstract:
The phonon dispersion was measured at room temperature along (0,0,L) in the tetragonal phase of LaFeAsO using inelastic x-ray scattering. Spin-polarized first-principles calculations imposing various types of antiferromagnetic order are in better agreement with the experimental results than nonmagnetic calculations, although the measurements were made well above the magnetic ordering temperature,…
▽ More
The phonon dispersion was measured at room temperature along (0,0,L) in the tetragonal phase of LaFeAsO using inelastic x-ray scattering. Spin-polarized first-principles calculations imposing various types of antiferromagnetic order are in better agreement with the experimental results than nonmagnetic calculations, although the measurements were made well above the magnetic ordering temperature, T_N. Splitting observed between two A_{1g} phonon modes at 22 and 26 meV is only observed in spin-polarized calculations. Magneto-structural effects similar to those observed in the AFe_2As_2 materials are confirmed present in LaFeAsO. The presence of Fe-spin is necessary to find reasonable agreement of the calculations with the measured spectrum well above T_N. On-site Fe and As force constants show significant softening compared to nonmagnetic calculations, however an investigation of the real-space force constants associates the magnetoelastic coupling with a complex renormalization instead of softening of a specific pairwise force.
△ Less
Submitted 5 June, 2012;
originally announced June 2012.
-
Unusual temperature dependence of band dispersion in Ba(Fe(1-x)Ru(x))2As2 and its consequences for antiferromagnetic ordering
Authors:
R. S. Dhaka,
S. E. Hahn,
E. Razzoli,
Rui Jiang,
M. Shi,
B. N. Harmon,
A. Thaler,
S. L. Bud'ko,
P. C. Canfield,
Adam Kaminski
Abstract:
We have performed detailed studies of the temperature evolution of the electronic structure in Ba(Fe(1-x)Ru(x))2As2 using Angle Resolved Photoemission Spectroscopy (ARPES). Surprisingly, we find that the binding energy of both hole and electron bands changes significantly with temperature in pure and Ru substituted samples. The hole and electron pockets are well nested at low temperature in unsubs…
▽ More
We have performed detailed studies of the temperature evolution of the electronic structure in Ba(Fe(1-x)Ru(x))2As2 using Angle Resolved Photoemission Spectroscopy (ARPES). Surprisingly, we find that the binding energy of both hole and electron bands changes significantly with temperature in pure and Ru substituted samples. The hole and electron pockets are well nested at low temperature in unsubstituted (BaFe2As2) samples, which likely drives the spin density wave (SDW) and resulting antiferromagnetic order. Upon warming, this nesting is degraded as the hole pocket shrinks and the electron pocket expands. Our results demonstrate that the temperature dependent nesting may play an important role in driving the antiferromagnetic/paramagnetic phase transition.
△ Less
Submitted 30 May, 2012;
originally announced May 2012.
-
Influence of Magnetism on Phonons in CaFe2As2 Via Inelastic X-ray Scattering
Authors:
S. E. Hahn,
Y. Lee,
N. Ni,
A. Alatas,
B. M. Leu,
D. Y. Chung,
I. S. Todorov,
E. E. Alp,
M. G. Kanatzidis,
P. C. Canfield,
A. I. Goldman,
R. J. McQueeney,
B. N. Harmon
Abstract:
In the iron pnictides, the strong sensitivity of the iron magnetic moment to the arsenic position suggests a significant relationship between phonons and magnetism. We measured the phonon dispersion of several branches in the high temperature tetragonal phase of CaFe2As2 using inelastic x-ray scattering on single-crystal samples. These measurements were compared to ab initio calculations of the…
▽ More
In the iron pnictides, the strong sensitivity of the iron magnetic moment to the arsenic position suggests a significant relationship between phonons and magnetism. We measured the phonon dispersion of several branches in the high temperature tetragonal phase of CaFe2As2 using inelastic x-ray scattering on single-crystal samples. These measurements were compared to ab initio calculations of the phonons. Spin polarized calculations imposing the antiferromagnetic order present in the low temperature orthorhombic phase dramatically improve agreement between theory and experiment. This is discussed in terms of the strong antiferromagnetic correlations that are known to persist in the tetragonal phase.
△ Less
Submitted 14 May, 2009; v1 submitted 28 February, 2009;
originally announced March 2009.