-
MTrainS: Improving DLRM training efficiency using heterogeneous memories
Authors:
Hiwot Tadese Kassa,
Paul Johnson,
Jason Akers,
Mrinmoy Ghosh,
Andrew Tulloch,
Dheevatsa Mudigere,
Jongsoo Park,
Xing Liu,
Ronald Dreslinski,
Ehsan K. Ardestani
Abstract:
Recommendation models are very large, requiring terabytes (TB) of memory during training. In pursuit of better quality, the model size and complexity grow over time, which requires additional training data to avoid overfitting. This model growth demands a large number of resources in data centers. Hence, training efficiency is becoming considerably more important to keep the data center power dema…
▽ More
Recommendation models are very large, requiring terabytes (TB) of memory during training. In pursuit of better quality, the model size and complexity grow over time, which requires additional training data to avoid overfitting. This model growth demands a large number of resources in data centers. Hence, training efficiency is becoming considerably more important to keep the data center power demand manageable. In Deep Learning Recommendation Models (DLRM), sparse features capturing categorical inputs through embedding tables are the major contributors to model size and require high memory bandwidth. In this paper, we study the bandwidth requirement and locality of embedding tables in real-world deployed models. We observe that the bandwidth requirement is not uniform across different tables and that embedding tables show high temporal locality. We then design MTrainS, which leverages heterogeneous memory, including byte and block addressable Storage Class Memory for DLRM hierarchically. MTrainS allows for higher memory capacity per node and increases training efficiency by lowering the need to scale out to multiple hosts in memory capacity bound use cases. By optimizing the platform memory hierarchy, we reduce the number of nodes for training by 4-8X, saving power and cost of training while meeting our target training performance.
△ Less
Submitted 19 April, 2023;
originally announced May 2023.
-
Technology-Enabled Disinformation: Summary, Lessons, and Recommendations
Authors:
John Akers,
Gagan Bansal,
Gabriel Cadamuro,
Christine Chen,
Quanze Chen,
Lucy Lin,
Phoebe Mulcaire,
Rajalakshmi Nandakumar,
Matthew Rockett,
Lucy Simko,
John Toman,
Tongshuang Wu,
Eric Zeng,
Bill Zorn,
Franziska Roesner
Abstract:
Technology is increasingly used -- unintentionally (misinformation) or intentionally (disinformation) -- to spread false information at scale, with potentially broad-reaching societal effects. For example, technology enables increasingly realistic false images and videos, and hyper-personal targeting means different people may see different versions of reality. This report is the culmination of a…
▽ More
Technology is increasingly used -- unintentionally (misinformation) or intentionally (disinformation) -- to spread false information at scale, with potentially broad-reaching societal effects. For example, technology enables increasingly realistic false images and videos, and hyper-personal targeting means different people may see different versions of reality. This report is the culmination of a PhD-level special topics course (https://courses.cs.washington.edu/courses/cse599b/18au/) in Computer Science & Engineering at the University of Washington's Paul G. Allen School in the fall of 2018. The goals of this course were to study (1) how technologies and today's technical platforms enable and support the creation and spread of such mis- and disinformation, as well as (2) how technical approaches could be used to mitigate these issues. In this report, we summarize the space of technology-enabled mis- and disinformation based on our investigations, and then surface our lessons and recommendations for technologists, researchers, platform designers, policymakers, and users.
△ Less
Submitted 3 January, 2019; v1 submitted 21 December, 2018;
originally announced December 2018.
-
The effects of resonant magnetic perturbations and charge-exchange reactions on fast ion confinement and neutron emission in the Mega Amp Spherical Tokamak
Authors:
K G McClements,
K Tani,
R J Akers,
Y Q Liu,
K Shinohara,
H Tsutsui,
S Tsuji-Iio
Abstract:
The application of non-axisymmetric resonant magnetic perturbations (RMPs) to low current plasmas in the Mega Amp Spherical Tokamak (MAST) was found to be correlated with substantial drops in neutron production, suggesting a significant degradation of fast ion confinement. The effects of such perturbations on fast ions in MAST have been modelled using a revised version of a non-steady-state orbit-…
▽ More
The application of non-axisymmetric resonant magnetic perturbations (RMPs) to low current plasmas in the Mega Amp Spherical Tokamak (MAST) was found to be correlated with substantial drops in neutron production, suggesting a significant degradation of fast ion confinement. The effects of such perturbations on fast ions in MAST have been modelled using a revised version of a non-steady-state orbit-following Monte-Carlo code (NSS OFMC), in which the parametrization of fusion reaction rates has been updated and neutron rates have been recalculated. Losses of fast ions via charge-exchange (CX) with background neutrals and the subsequent reionization of fast neutrals due to collisions with bulk plasma particles have also been taken into account. The effects of the plasma response to externally-applied RMPs have been included in the modelling. The updated results show that computed neutron rates in the presence of RMPs with the plasma response and CX reactions taken into account agree very well with the experimental data throughout the analysis target time. CX reactions play an important role in determining the neutron rates, in particular before the onset of RMPs.
△ Less
Submitted 15 June, 2018;
originally announced June 2018.
-
Measurements and modelling of fast-ion redistribution due to resonant MHD instabilities in MAST
Authors:
O. M. Jones,
M. Cecconello,
K. G. McClements,
I. Klimek,
R. J. Akers,
W. U. Boeglin,
D. L. Keeling,
A. J. Meakins,
R. V. Perez,
S. E. Sharapov,
M. Turnyanskiy
Abstract:
The results of a comprehensive investigation into the effects of toroidicity-induced Alfvén eigenmodes (TAE) and energetic particle modes on the NBI-generated fast-ion population in MAST plasmas are reported. Fast-ion redistribution due to frequency-chir** TAE in the range 50 kHz to 100 kHz, and frequency-chir** energetic particle modes known as fishbones in the range 20 kHz to 50 kHz, is obse…
▽ More
The results of a comprehensive investigation into the effects of toroidicity-induced Alfvén eigenmodes (TAE) and energetic particle modes on the NBI-generated fast-ion population in MAST plasmas are reported. Fast-ion redistribution due to frequency-chir** TAE in the range 50 kHz to 100 kHz, and frequency-chir** energetic particle modes known as fishbones in the range 20 kHz to 50 kHz, is observed. TAE and fishbones are also observed to cause losses of fast ions from the plasma. The spatial and temporal evolution of the fast-ion distribution is determined using a fission chamber, a radially-scanning collimated neutron flux monitor, a fast-ion deuterium alpha spectrometer and a charged fusion product detector. Modelling using the global transport analysis code TRANSP, with ad hoc anomalous diffusion and fishbone loss models introduced, reproduces the coarsest features of the affected fast-ion distribution in the presence of energetic-particle-driven modes. The spectrally and spatially resolved measurements show however that these models do not fully capture the effects of chir** modes on the fast-ion distribution.
△ Less
Submitted 23 March, 2015;
originally announced March 2015.
-
Measurements and modelling of fast-ion redistribution due to MHD instabilities in the Mega-Amp Spherical Tokamak
Authors:
O M Jones,
C A Michael,
M Cecconello,
I Wodniak,
K G McClements,
D L Keeling,
C D Challis,
M Turnyanskiy,
A J Meakins,
N J Conway,
B J Crowley,
R J Akers,
MAST team
Abstract:
The results of a comprehensive investigation into the effects of various MHD modes on the NBI-generated fast-ion population in MAST plasmas are reported. Fast-ion redistribution due to low-frequency (20-50 kHz) chir** energetic particle modes known as fishbones, as well as the long-lived internal kink mode, is observed with the Fast-Ion Deuterium Alpha (FIDA) spectrometer and radially-scanning c…
▽ More
The results of a comprehensive investigation into the effects of various MHD modes on the NBI-generated fast-ion population in MAST plasmas are reported. Fast-ion redistribution due to low-frequency (20-50 kHz) chir** energetic particle modes known as fishbones, as well as the long-lived internal kink mode, is observed with the Fast-Ion Deuterium Alpha (FIDA) spectrometer and radially-scanning collimated neutron camera. In addition, strongly-driven chir** toroidicity-induced Alfven eigenmodes are observed to cause fast-ion redistribution, as are sawteeth and large edge-localised modes. In each case, the modes affect fast ions in a region of real space governed by the eigenmode structure and principal toroidal mode number. Modelling using the global transport analysis code TRANSP, with ad hoc anomalous diffusion introduced, reproduces the coarsest features of the affected fast-ion distribution in the presence of energetic-particle-driven modes, but the spectrally and spatially resolved FIDA measurements suggest that the distribution exhibits structure on a finer scale than is accounted for by this model.
△ Less
Submitted 27 January, 2014;
originally announced January 2014.
-
Measurement and control of the fast ion redistribution on MAST
Authors:
M. Turnyanskiy,
C. D. Challis,
R. J. Akers,
M. Cecconello,
D. L. Keeling,
A. Kirk,
R. Lake,
S. D. Pinches,
S. Sangaroon,
I. Wodniak
Abstract:
Previous experiments on MAST and other tokamaks have indicated that the level of fast ion redistribution can exceed that expected from classical diffusion and that this level increases with beam power. In this paper we present a quantification of this effect in MAST plasmas using a recently commissioned scanning neutron camera. The observed fast ion diffusivity correlates with the amplitude of n=1…
▽ More
Previous experiments on MAST and other tokamaks have indicated that the level of fast ion redistribution can exceed that expected from classical diffusion and that this level increases with beam power. In this paper we present a quantification of this effect in MAST plasmas using a recently commissioned scanning neutron camera. The observed fast ion diffusivity correlates with the amplitude of n=1 energetic particle modes, indicating that they are the probable cause of the non-classical fast ion diffusion in MAST. Finally, it will be shown that broadening the fast ion pressure profile by the application of neutral beam injection at an off-axis location can mitigate the growth of these modes and result in the classical fast ion behaviour
△ Less
Submitted 1 July, 2013;
originally announced July 2013.
-
Fast-Ion Deuterium Alpha spectroscopic observations of the effects of fishbones in the Mega-Ampere Spherical Tokamak
Authors:
O. M. Jones,
C. A. Michael,
K. G. McClements,
N. J. Conway,
B. Crowley,
R. J. Akers,
R. J. Lake,
S. D. Pinches
Abstract:
Using the recently-installed Fast-Ion Deuterium Alpha (FIDA) spectrometer, the effects of low-frequency (20-50 kHz) chir** energetic particle modes with toroidal mode number n \geq 1 on the fast-ion population in MAST plasmas are considered. Results from the FIDA diagnostic are presented and discussed in the light of the present theoretical understanding of these modes, known as fishbones, in pl…
▽ More
Using the recently-installed Fast-Ion Deuterium Alpha (FIDA) spectrometer, the effects of low-frequency (20-50 kHz) chir** energetic particle modes with toroidal mode number n \geq 1 on the fast-ion population in MAST plasmas are considered. Results from the FIDA diagnostic are presented and discussed in the light of the present theoretical understanding of these modes, known as fishbones, in plasmas with reversed shear. Measurements of the fast-ion population reveal strong redistribution of fast ions in both real and velocity space as a result of the fishbones. Time-resolved measurements throughout the evolution of a fishbone show radial redistribution of fast ions with energies up to 95% of the primary beam injection energy. Correlations between changes in the FIDA signal and the peak time derivative of the magnetic field perturbation are observed in a limited range of operating scenarios. The transient reduction in signal caused by a fishbone may in some cases reach 50% of the signal intensity before mode onset.
△ Less
Submitted 24 April, 2013; v1 submitted 4 April, 2013;
originally announced April 2013.