-
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models
Authors:
Nastaran Saadati,
Minh Pham,
Nasla Saleem,
Joshua R. Waite,
Aditya Balu,
Zhanhong Jiang,
Chinmay Hegde,
Soumik Sarkar
Abstract:
Recent advances in decentralized deep learning algorithms have demonstrated cutting-edge performance on various tasks with large pre-trained models. However, a pivotal prerequisite for achieving this level of competitiveness is the significant communication and computation overheads when updating these models, which prohibits the applications of them to real-world scenarios. To address this issue,…
▽ More
Recent advances in decentralized deep learning algorithms have demonstrated cutting-edge performance on various tasks with large pre-trained models. However, a pivotal prerequisite for achieving this level of competitiveness is the significant communication and computation overheads when updating these models, which prohibits the applications of them to real-world scenarios. To address this issue, drawing inspiration from advanced model merging techniques without requiring additional training, we introduce the Decentralized Iterative Merging-And-Training (DIMAT) paradigm--a novel decentralized deep learning framework. Within DIMAT, each agent is trained on their local data and periodically merged with their neighboring agents using advanced model merging techniques like activation matching until convergence is achieved. DIMAT provably converges with the best available rate for nonconvex functions with various first-order methods, while yielding tighter error bounds compared to the popular existing approaches. We conduct a comprehensive empirical analysis to validate DIMAT's superiority over baselines across diverse computer vision tasks sourced from multiple datasets. Empirical results validate our theoretical claims by showing that DIMAT attains faster and higher initial gain in accuracy with independent and identically distributed (IID) and non-IID data, incurring lower communication overhead. This DIMAT paradigm presents a new opportunity for the future decentralized learning, enhancing its adaptability to real-world with sparse and light-weight communication and computation.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Active shooter detection and robust tracking utilizing supplemental synthetic data
Authors:
Joshua R. Waite,
Jiale Feng,
Riley Tavassoli,
Laura Harris,
Sin Yong Tan,
Subhadeep Chakraborty,
Soumik Sarkar
Abstract:
The increasing concern surrounding gun violence in the United States has led to a focus on develo** systems to improve public safety. One approach to develo** such a system is to detect and track shooters, which would help prevent or mitigate the impact of violent incidents. In this paper, we proposed detecting shooters as a whole, rather than just guns, which would allow for improved tracking…
▽ More
The increasing concern surrounding gun violence in the United States has led to a focus on develo** systems to improve public safety. One approach to develo** such a system is to detect and track shooters, which would help prevent or mitigate the impact of violent incidents. In this paper, we proposed detecting shooters as a whole, rather than just guns, which would allow for improved tracking robustness, as obscuring the gun would no longer cause the system to lose sight of the threat. However, publicly available data on shooters is much more limited and challenging to create than a gun dataset alone. Therefore, we explore the use of domain randomization and transfer learning to improve the effectiveness of training with synthetic data obtained from Unreal Engine environments. This enables the model to be trained on a wider range of data, increasing its ability to generalize to different situations. Using these techniques with YOLOv8 and Deep OC-SORT, we implemented an initial version of a shooter tracking system capable of running on edge hardware, including both a Raspberry Pi and a Jetson Nano.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
Spin-density-wave order controlled by uniaxial stress in CeAuSb$_2$
Authors:
R. Waite,
F. Orlandi,
D. A. Sokolov,
R. A. Ribeiro,
P. C. Canfield,
P. Manuel,
D. D. Khalyavin,
C. W. Hicks,
S. M. Hayden
Abstract:
The tetragonal heavy-fermion compound CeAuSb$_2$ (space group $P4/nmm$) exhibits incommensurate spin density wave (SDW) order below $T_{N}\approx6.5~K$ with the propagation vector $\mathbf{q}_A = (δ_A,δ_A,1/2)$. The application of uniaxial stress along the [010] direction induces a sudden change in the resistivity ratio $ρ_a/ρ_b$ at a compressive strain of $ε\approx -0.5$\%. Here we use neutron sc…
▽ More
The tetragonal heavy-fermion compound CeAuSb$_2$ (space group $P4/nmm$) exhibits incommensurate spin density wave (SDW) order below $T_{N}\approx6.5~K$ with the propagation vector $\mathbf{q}_A = (δ_A,δ_A,1/2)$. The application of uniaxial stress along the [010] direction induces a sudden change in the resistivity ratio $ρ_a/ρ_b$ at a compressive strain of $ε\approx -0.5$\%. Here we use neutron scattering to show that the uniaxial stress induces a first-order transition to a SDW state with a different propagation vector $(0,δ_B,1/2)$ with $δ_B=0.25$. The magnetic structure of the new (B) phase consists of Ce layers with ordered moments alternating with layers with zero moment stacked along the $c$-axis. The ordered layers have an up-up-down-down configuration along the $b$-axis. This is an unusual situation in which the loss of spatial inversion is driven by the magnetic order. We argue that the change in SDW wavevector leads to Fermi surface reconstruction and a concomitant change in the transport properties.
△ Less
Submitted 31 July, 2023; v1 submitted 23 February, 2022;
originally announced February 2022.
-
Heisenberg spins on an anisotropic triangular lattice: PdCrO2 under uniaxial stress
Authors:
Dan Sun,
Dmitry A. Sokolov,
Richard Waite,
Seunghyun Khim,
Pascal Manuel,
Fabio Orlandi,
Dmitry D. Khalyavin,
Andrew P. Mackenzie,
Clifford W. Hicks
Abstract:
When Heisenberg spins interact antiferromagnetically on a triangular lattice and nearest-neighbor interactions dominate, the ground state is 120$^{\circ}$ antiferromagnetism. In this work, we probe the response of this state to lifting the triangular symmetry, through investigation of the triangular antiferromagnet PdCrO$_2$ under uniaxial stress by neutron diffraction and resistivity measurements…
▽ More
When Heisenberg spins interact antiferromagnetically on a triangular lattice and nearest-neighbor interactions dominate, the ground state is 120$^{\circ}$ antiferromagnetism. In this work, we probe the response of this state to lifting the triangular symmetry, through investigation of the triangular antiferromagnet PdCrO$_2$ under uniaxial stress by neutron diffraction and resistivity measurements. The periodicity of the magnetic order is found to change rapidly with applied stress; the rate of change indicates that the magnetic anisotropy is roughly forty times the stress-induced bond length anisotropy. At low stress, the incommensuration period becomes extremely long, on the order of 1000 lattice spacings; no locking of the magnetism to commensurate periodicity is detected. Separately, the magnetic structure is found to undergo a first-order transition at a compressive stress of $\sim$0.4 GPa, at which the interlayer ordering switches from a double- to a single-q structure.
△ Less
Submitted 19 December, 2021;
originally announced December 2021.
