Showing 1–24 of 24 results for author: Patel, K K

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication

Authors: Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari, Lingxiao Wang, Sebastian U. Stich, Ziheng Cheng, Nirmit Joshi, Nathan Srebro

Abstract: Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically proving the dominance of local SGD in settings with reasonable data heterogeneity has been difficult, creating a significant gap between theory and practice. In this paper, we provide new lower bounds for local SGD under… ▽ More Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically proving the dominance of local SGD in settings with reasonable data heterogeneity has been difficult, creating a significant gap between theory and practice. In this paper, we provide new lower bounds for local SGD under existing first-order data heterogeneity assumptions, showing that these assumptions are insufficient to prove the effectiveness of local update steps. Furthermore, under these same assumptions, we demonstrate the min-max optimality of accelerated mini-batch SGD, which fully resolves our understanding of distributed optimization for several problem classes. Our results emphasize the need for better models of data heterogeneity to understand the effectiveness of local SGD in practice. Towards this end, we consider higher-order smoothness and heterogeneity assumptions, providing new upper bounds that imply the dominance of local SGD over mini-batch SGD when data heterogeneity is low. △ Less

Submitted 19 May, 2024; originally announced May 2024.

Critical slowing of the spin and charge density wave order in thin film Cr following photoexcitation

Authors: Sheena K. K. Patel, Oleg Yu. Gorobtsov, Devin Cela, Stjepan B. Hrkac, Nelson Hua, Rajasekhar Medapalli, Anatoly G. Shabalin, James Wingert, James M. Glownia, Diling Zhu, Matthieu Chollet, Oleg G. Shpyrko, Andrej Singer, Eric E. Fullerton

Abstract: We report on the evolution of the charge density wave (CDW) and spin density wave (SDW) order of a chromium film following photoexcitation with an ultrafast optical laser pulse. The CDW is measured by ultrafast time-resolved x-ray diffraction of the CDW satellite that tracks the suppression and recovery of the CDW following photoexcitation. We find that as the temperature of the film approaches a… ▽ More We report on the evolution of the charge density wave (CDW) and spin density wave (SDW) order of a chromium film following photoexcitation with an ultrafast optical laser pulse. The CDW is measured by ultrafast time-resolved x-ray diffraction of the CDW satellite that tracks the suppression and recovery of the CDW following photoexcitation. We find that as the temperature of the film approaches a discontinuous phase transition in the CDW and SDW order, the time scales of recovery increase exponentially from the expected thermal time scales. We extend a Landau model for SDW systems to account for this critical slowing with the appropriate boundary conditions imposed by the geometry of the thin film system. This model allows us to assess the energy barrier between available CDW/SDW states with different spatial periodicities. △ Less

Submitted 5 March, 2024; v1 submitted 29 February, 2024; originally announced March 2024.

Comments: Author typo fixed

Federated Online and Bandit Convex Optimization

Authors: Kumar Kshitij Patel, Lingxiao Wang, Aadirupa Saha, Nati Sebro

Abstract: We study the problems of distributed online and bandit convex optimization against an adaptive adversary. We aim to minimize the average regret on $M$ machines working in parallel over $T$ rounds with $R$ intermittent communications. Assuming the underlying cost functions are convex and can be generated adaptively, our results show that collaboration is not beneficial when the machines have access… ▽ More We study the problems of distributed online and bandit convex optimization against an adaptive adversary. We aim to minimize the average regret on $M$ machines working in parallel over $T$ rounds with $R$ intermittent communications. Assuming the underlying cost functions are convex and can be generated adaptively, our results show that collaboration is not beneficial when the machines have access to the first-order gradient information at the queried points. This is in contrast to the case for stochastic functions, where each machine samples the cost functions from a fixed distribution. Furthermore, we delve into the more challenging setting of federated online optimization with bandit (zeroth-order) feedback, where the machines can only access values of the cost functions at the queried points. The key finding here is identifying the high-dimensional regime where collaboration is beneficial and may even lead to a linear speedup in the number of machines. We further illustrate our findings through federated adversarial linear bandits by develo** novel distributed single and two-point feedback algorithms. Our work is the first attempt towards a systematic understanding of federated online optimization with limited feedback, and it attains tight regret bounds in the intermittent communication setting for both first and zeroth-order feedback. Our results thus bridge the gap between stochastic and adaptive settings in federated online optimization. △ Less

Submitted 29 November, 2023; originally announced November 2023.

On the Effect of Defections in Federated Learning and How to Prevent Them

Authors: Minbiao Han, Kumar Kshitij Patel, Han Shao, Lingxiao Wang

Abstract: Federated learning is a machine learning protocol that enables a large population of agents to collaborate over multiple rounds to produce a single consensus model. There are several federated learning applications where agents may choose to defect permanently$-$essentially withdrawing from the collaboration$-$if they are content with their instantaneous model in that round. This work demonstrates… ▽ More Federated learning is a machine learning protocol that enables a large population of agents to collaborate over multiple rounds to produce a single consensus model. There are several federated learning applications where agents may choose to defect permanently$-$essentially withdrawing from the collaboration$-$if they are content with their instantaneous model in that round. This work demonstrates the detrimental impact of such defections on the final model's robustness and ability to generalize. We also show that current federated optimization algorithms fail to disincentivize these harmful defections. We introduce a novel optimization algorithm with theoretical guarantees to prevent defections while ensuring asymptotic convergence to an effective solution for all participating agents. We also provide numerical experiments to corroborate our findings and demonstrate the effectiveness of our algorithm. △ Less

Submitted 27 November, 2023; originally announced November 2023.

A rigorous benchmarking of methods for SARS-CoV-2 lineage abundance estimation in wastewater

Authors: Viorel Munteanu, Victor Gordeev, Michael Saldana, Eva Aßmann, Justin Maine Su, Nicolae Drabcinski, Oksana Zlenko, Maryna Kit, Felicia Iordachi, Khooshbu Kantibhai Patel, Abdullah Al Nahid, Likhitha Chittampalli, Yidian Xu, Pavel Skums, Shelesh Agrawal, Martin Hölzer, Adam Smith, Alex Zelikovsky, Serghei Mangul

Abstract: In light of the continuous transmission and evolution of SARS-CoV-2 coupled with a significant decline in clinical testing, there is a pressing need for scalable, cost-effective, long-term, passive surveillance tools to effectively monitor viral variants circulating in the population. Wastewater genomic surveillance of SARS-CoV-2 has arrived as an alternative to clinical genomic surveillance, allo… ▽ More In light of the continuous transmission and evolution of SARS-CoV-2 coupled with a significant decline in clinical testing, there is a pressing need for scalable, cost-effective, long-term, passive surveillance tools to effectively monitor viral variants circulating in the population. Wastewater genomic surveillance of SARS-CoV-2 has arrived as an alternative to clinical genomic surveillance, allowing to continuously monitor the prevalence of viral lineages in communities of various size at a fraction of the time, cost, and logistic effort and serving as an early warning system for emerging variants, critical for developed communities and especially for underserved ones. Importantly, lineage prevalence estimates obtained with this approach aren't distorted by biases related to clinical testing accessibility and participation. However, the relative performance of bioinformatics methods used to measure relative lineage abundances from wastewater sequencing data is unknown, preventing both the research community and public health authorities from making informed decisions regarding computational tool selection. Here, we perform comprehensive benchmarking of 18 bioinformatics methods for estimating the relative abundance of SARS-CoV-2 (sub)lineages in wastewater by using data from 36 in vitro mixtures of synthetic lineage and sublineage genomes. In addition, we use simulated data from 78 mixtures of lineages and sublineages co-occurring in the clinical setting with proportions mirroring their prevalence ratios observed in real data. Importantly, we investigate how the accuracy of the evaluated methods is impacted by the sequencing technology used, the associated error rate, the read length, read depth, but also by the exposure of the synthetic RNA mixtures to wastewater, with the goal of capturing the effects induced by the wastewater matrix, including RNA fragmentation and degradation. △ Less

Submitted 21 January, 2024; v1 submitted 29 September, 2023; originally announced September 2023.

Comments: For correspondence: [email protected]

An improved column-generation-based matheuristic for learning classification trees

Authors: Krunal Kishor Patel, Guy Desaulniers, Andrea Lodi

Abstract: Decision trees are highly interpretable models for solving classification problems in machine learning (ML). The standard ML algorithms for training decision trees are fast but generate suboptimal trees in terms of accuracy. Other discrete optimization models in the literature address the optimality problem but only work well on relatively small datasets. \cite{firat2020column} proposed a column-g… ▽ More Decision trees are highly interpretable models for solving classification problems in machine learning (ML). The standard ML algorithms for training decision trees are fast but generate suboptimal trees in terms of accuracy. Other discrete optimization models in the literature address the optimality problem but only work well on relatively small datasets. \cite{firat2020column} proposed a column-generation-based heuristic approach for learning decision trees. This approach improves scalability and can work with large datasets. In this paper, we describe improvements to this column generation approach. First, we modify the subproblem model to significantly reduce the number of subproblems in multiclass classification instances. Next, we show that the data-dependent constraints in the master problem are implied, and use them as cutting planes. Furthermore, we describe a separation model to generate data points for which the linear programming relaxation solution violates their corresponding constraints. We conclude by presenting computational results that show that these modifications result in better scalability. △ Less

Submitted 22 January, 2024; v1 submitted 22 August, 2023; originally announced August 2023.

Comments: Submitted to Computers and Operations Research journal

Progressively Strengthening and Tuning MIP Solvers for Reoptimization

Authors: Krunal Kishor Patel

Abstract: This paper explores reoptimization techniques for solving sequences of similar mixed integer programs (MIPs) more effectively. Traditionally, these MIPs are solved independently, without capitalizing on information from previously solved instances. Our approach focuses on primal bound improvements by reusing the solutions of the previously solved instances, as well as dual bound improvements by re… ▽ More This paper explores reoptimization techniques for solving sequences of similar mixed integer programs (MIPs) more effectively. Traditionally, these MIPs are solved independently, without capitalizing on information from previously solved instances. Our approach focuses on primal bound improvements by reusing the solutions of the previously solved instances, as well as dual bound improvements by reusing the branching history and automating parameter tuning. We also describe ways to improve the solver performance by extending ideas from reliability branching to generate better pseudocosts. Our reoptimization approach, crafted for the MIP 2023 workshop computational competition, was honored with the first prize. In this paper, we thoroughly analyze the performance of each technique and their combined impact on the solver's performance. Finally, we present ways to extend our techniques in practice for further improvements. △ Less

Submitted 25 January, 2024; v1 submitted 17 August, 2023; originally announced August 2023.

Comments: Submitted to MPC journal

MSC Class: 90C11 (Primary); 90C10; 90-08 (Secondary)

Explainable prediction of Qcodes for NOTAMs using column generation

Authors: Krunal Kishor Patel, Guy Desaulniers, Andrea Lodi, Freddy Lecue

Abstract: A NOtice To AirMen (NOTAM) contains important flight route related information. To search and filter them, NOTAMs are grouped into categories called QCodes. In this paper, we develop a tool to predict, with some explanations, a Qcode for a NOTAM. We present a way to extend the interpretable binary classification using column generation proposed in Dash, Gunluk, and Wei (2018) to a multiclass text… ▽ More A NOtice To AirMen (NOTAM) contains important flight route related information. To search and filter them, NOTAMs are grouped into categories called QCodes. In this paper, we develop a tool to predict, with some explanations, a Qcode for a NOTAM. We present a way to extend the interpretable binary classification using column generation proposed in Dash, Gunluk, and Wei (2018) to a multiclass text classification method. We describe the techniques used to tackle the issues related to one vs-rest classification, such as multiple outputs and class imbalances. Furthermore, we introduce some heuristics, including the use of a CP-SAT solver for the subproblems, to reduce the training time. Finally, we show that our approach compares favorably with state-of-the-art machine learning algorithms like Linear SVM and small neural networks while adding the needed interpretability component. △ Less

Submitted 20 January, 2023; v1 submitted 9 August, 2022; originally announced August 2022.

Ultrafast Emergence of Ferromagnetism in Antiferromagnetic FeRh in High Magnetic Fields

Authors: I. A. Dolgikh, T. G. H. Blank, A. G. Buzdakov, G. Li, K. H. Prabhakara, S. K. K. Patel, R. Medapalli, E. E. Fullerton, O. V. Koplak, J. H. Mentink, K. A. Zvezdin, A. K. Zvezdin, P. C. M. Christianen, A. V. Kimel

Abstract: Ultrafast heating of FeRh by a femtosecond laser pulse launches a magneto-structural phase transition from an antiferromagnetic to a ferromagnetic state. Aiming to reveal the ultrafast kinetics of this transition, we studied magnetization dynamics with the help of the magneto-optical Kerr effect in a broad range of temperatures (from 4 K to 400 K) and magnetic fields (up to 25 T). Three different… ▽ More Ultrafast heating of FeRh by a femtosecond laser pulse launches a magneto-structural phase transition from an antiferromagnetic to a ferromagnetic state. Aiming to reveal the ultrafast kinetics of this transition, we studied magnetization dynamics with the help of the magneto-optical Kerr effect in a broad range of temperatures (from 4 K to 400 K) and magnetic fields (up to 25 T). Three different types of ultrafast magnetization dynamics were observed and, using a numerically calculated H-T phase diagram, the differences were explained by different initial states of FeRh corresponding to a (i) collinear antiferromagnetic, (ii) canted antiferromagnetic and (iii) ferromagnetic alignment of spins. We argue that ultrafast heating of FeRh in the canted antiferromagnetic phase launches practically the fastest possible emergence of magnetization in this material. The magnetization emerges on a time scale of 2 ps, which corresponds to the earlier reported time-scale of the structural changes during the phase transition. △ Less

Submitted 27 July, 2023; v1 submitted 8 February, 2022; originally announced February 2022.

Megahertz-rate Ultrafast X-ray Scattering and Holographic Imaging at the European XFEL

Authors: Nanna Zhou Hagström, Michael Schneider, Nico Kerber, Alexander Yaroslavtsev, Erick Burgos Parra, Marijan Beg, Martin Lang, Christian M. Günther, Boris Seng, Fabian Kammerbauer, Horia Popescu, Matteo Pancaldi, Kumar Neeraj, Debanjan Polley, Rahul Jangid, Stjepan B. Hrkac, Sheena K. K. Patel, Sergei Ovcharenko, Diego Turenne, Dmitriy Ksenzov, Christine Boeglin, Igor Pronin, Marina Baidakova, Clemens von Korff Schmising, Martin Borchert , et al. (75 additional authors not shown)

Abstract: The advent of X-ray free-electron lasers (XFELs) has revolutionized fundamental science, from atomic to condensed matter physics, from chemistry to biology, giving researchers access to X-rays with unprecedented brightness, coherence, and pulse duration. All XFEL facilities built until recently provided X-ray pulses at a relatively low repetition rate, with limited data statistics. Here, we presen… ▽ More The advent of X-ray free-electron lasers (XFELs) has revolutionized fundamental science, from atomic to condensed matter physics, from chemistry to biology, giving researchers access to X-rays with unprecedented brightness, coherence, and pulse duration. All XFEL facilities built until recently provided X-ray pulses at a relatively low repetition rate, with limited data statistics. Here, we present the results from the first megahertz repetition rate X-ray scattering experiments at the Spectroscopy and Coherent Scattering (SCS) instrument of the European XFEL. We illustrate the experimental capabilities that the SCS instrument offers, resulting from the operation at MHz repetition rates and the availability of the novel DSSC 2D imaging detector. Time-resolved magnetic X-ray scattering and holographic imaging experiments in solid state samples were chosen as representative, providing an ideal test-bed for operation at megahertz rates. Our results are relevant and applicable to any other non-destructive XFEL experiments in the soft X-ray range. △ Less

Submitted 20 January, 2022; v1 submitted 17 January, 2022; originally announced January 2022.

Comments: 13 pages, 5 figures. Supplementary Information as ancillary file

Journal ref: J. Synchrotron Rad. (2022), 29

A real-time spatiotemporal AI model analyzes skill in open surgical videos

Authors: Emmett D. Goodman, Krishna K. Patel, Yilun Zhang, William Locke, Chris J. Kennedy, Rohan Mehrotra, Stephen Ren, Melody Y. Guan, Maren Downing, Hao Wei Chen, Jevin Z. Clark, Gabriel A. Brat, Serena Yeung

Abstract: Open procedures represent the dominant form of surgery worldwide. Artificial intelligence (AI) has the potential to optimize surgical practice and improve patient outcomes, but efforts have focused primarily on minimally invasive techniques. Our work overcomes existing data limitations for training AI models by curating, from YouTube, the largest dataset of open surgical videos to date: 1997 video… ▽ More Open procedures represent the dominant form of surgery worldwide. Artificial intelligence (AI) has the potential to optimize surgical practice and improve patient outcomes, but efforts have focused primarily on minimally invasive techniques. Our work overcomes existing data limitations for training AI models by curating, from YouTube, the largest dataset of open surgical videos to date: 1997 videos from 23 surgical procedures uploaded from 50 countries. Using this dataset, we developed a multi-task AI model capable of real-time understanding of surgical behaviors, hands, and tools - the building blocks of procedural flow and surgeon skill. We show that our model generalizes across diverse surgery types and environments. Illustrating this generalizability, we directly applied our YouTube-trained model to analyze open surgeries prospectively collected at an academic medical center and identified kinematic descriptors of surgical skill related to efficiency of hand motion. Our Annotated Videos of Open Surgery (AVOS) dataset and trained model will be made available for further development of surgical AI. △ Less

Submitted 14 December, 2021; originally announced December 2021.

Comments: 22 pages, 4 main text figures, 7 extended data figures, 4 extended data tables

A Stochastic Newton Algorithm for Distributed Convex Optimization

Authors: Brian Bullins, Kumar Kshitij Patel, Ohad Shamir, Nathan Srebro, Blake Woodworth

Abstract: We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same population objective, as well as stochastic Hessian-vector products (products of an independent unbiased estimator of the Hessian of the population objective with arbitrary vectors), with many such stochastic computations… ▽ More We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same population objective, as well as stochastic Hessian-vector products (products of an independent unbiased estimator of the Hessian of the population objective with arbitrary vectors), with many such stochastic computations performed between rounds of communication. We show that our method can reduce the number, and frequency, of required communication rounds compared to existing methods without hurting performance, by proving convergence guarantees for quasi-self-concordant objectives (e.g., logistic regression), alongside empirical evidence. △ Less

Submitted 7 October, 2021; originally announced October 2021.

Phonon-assisted formation of an itinerant electronic density wave

Authors: Jiaruo Li, Oleg Yu. Gorobtsov, Sheena K. K. Patel, Nelson Hua, Benjamin Gregory, Anatoly G. Shabalin, Stjepan Hrkac, James Wingert, Devin Cela, James M. Glownia, Matthieu Chollet, Diling Zhu, Rajasekhar Medapalli, Eric E. Fullerton, Oleg G. Shpyrko, Andrej Singer

Abstract: Electronic instabilities drive ordering transitions in condensed matter. Despite many advances in the microscopic understanding of the ordered states, a more nuanced and profound question often remains unanswered: how do the collective excitations influence the electronic order formation? Here, we experimentally show that a phonon affects the spin density wave (SDW) formation after an SDW-quench b… ▽ More Electronic instabilities drive ordering transitions in condensed matter. Despite many advances in the microscopic understanding of the ordered states, a more nuanced and profound question often remains unanswered: how do the collective excitations influence the electronic order formation? Here, we experimentally show that a phonon affects the spin density wave (SDW) formation after an SDW-quench by femtosecond laser pulses. In a thin film, the temperature-dependent SDW period is quantized, allowing us to track the out-of-equilibrium formation path of the SDW precisely. By exploiting its persistent coupling to the lattice, we probe the SDW through the transient lattice distortion, measured by femtosecond X-ray diffraction. We find that within 500 femtoseconds after a complete quench, the SDW forms with the low-temperature period, directly bypassing a thermal state with the high-temperature period. We argue that a momentum-matched phonon launched by the quench changes the formation path of the SDW through the dynamic pinning of the order parameter. △ Less

Submitted 9 December, 2020; originally announced December 2020.

Minibatch vs Local SGD for Heterogeneous Distributed Learning

Authors: Blake Woodworth, Kumar Kshitij Patel, Nathan Srebro

Abstract: We analyze Local SGD (aka parallel or federated SGD) and Minibatch SGD in the heterogeneous distributed setting, where each machine has access to stochastic gradient estimates for a different, machine-specific, convex objective; the goal is to optimize w.r.t. the average objective; and machines can only communicate intermittently. We argue that, (i) Minibatch SGD (even without acceleration) domina… ▽ More We analyze Local SGD (aka parallel or federated SGD) and Minibatch SGD in the heterogeneous distributed setting, where each machine has access to stochastic gradient estimates for a different, machine-specific, convex objective; the goal is to optimize w.r.t. the average objective; and machines can only communicate intermittently. We argue that, (i) Minibatch SGD (even without acceleration) dominates all existing analysis of Local SGD in this setting, (ii) accelerated Minibatch SGD is optimal when the heterogeneity is high, and (iii) present the first upper bound for Local SGD that improves over Minibatch SGD in a non-homogeneous regime. △ Less

Submitted 1 March, 2022; v1 submitted 8 June, 2020; originally announced June 2020.

Comments: 34 pages

Femtosecond Photocurrents at the Pt/FeRh Interface

Authors: Rajasekhar Medapalli, Guanqiao Li, Sheena K. K. Patel, Rostislav. V. Mikhaylovskiy, Theo Rasing, Alexey V. Kimel, Eric E. Fullerton

Abstract: Femtosecond laser excitation of FeRh/Pt bilayers launches an ultrafast pulse of electric photocurrent in the Pt-layer and thus results in emission of electromagnetic radiation in the THz spectral range. Analysis of the THz emission as a function of polarization of the femtosecond laser pulse, external magnetic field, sample temperature and sample orientation shows that photocurrent can emerge due… ▽ More Femtosecond laser excitation of FeRh/Pt bilayers launches an ultrafast pulse of electric photocurrent in the Pt-layer and thus results in emission of electromagnetic radiation in the THz spectral range. Analysis of the THz emission as a function of polarization of the femtosecond laser pulse, external magnetic field, sample temperature and sample orientation shows that photocurrent can emerge due to vertical spin pum** and photo-induced inverse spin-orbit torque at the FeRh/Pt interface. The vertical spin pum** from FeRh to Pt does not depend on the polarization of light and originates from ultrafast laser-induced demagnetization of the ferromagnetic phase of FeRh. The photo-induced inverse spin-orbit torque at the FeRh/Pt interface can be described in terms of a helicity-dependent effect of circularly polarized light on the magnetization of the ferromagnetic FeRh and subsequent generation of a photocurrent. △ Less

Submitted 27 May, 2020; originally announced May 2020.

Comments: 11 pages

Is Local SGD Better than Minibatch SGD?

Authors: Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, Nathan Srebro

Abstract: We study local SGD (also known as parallel SGD and federated averaging), a natural and frequently used stochastic distributed optimization method. Its theoretical foundations are currently lacking and we highlight how all existing error guarantees in the convex setting are dominated by a simple baseline, minibatch SGD. (1) For quadratic objectives we prove that local SGD strictly dominates minibat… ▽ More We study local SGD (also known as parallel SGD and federated averaging), a natural and frequently used stochastic distributed optimization method. Its theoretical foundations are currently lacking and we highlight how all existing error guarantees in the convex setting are dominated by a simple baseline, minibatch SGD. (1) For quadratic objectives we prove that local SGD strictly dominates minibatch SGD and that accelerated local SGD is minimax optimal for quadratics; (2) For general convex objectives we provide the first guarantee that at least sometimes improves over minibatch SGD; (3) We show that indeed local SGD does not dominate minibatch SGD by presenting a lower bound on the performance of local SGD that is worse than the minibatch SGD guarantee. △ Less

Submitted 20 July, 2020; v1 submitted 18 February, 2020; originally announced February 2020.

Comments: 29 pages

Direct measurement of temporal correlations above the spin-glass transition by coherent resonant magnetic x-ray spectroscopy

Authors: **g** Song, Sheena K. K. Patel, Rupak Bhattacharya, Yi Yang, Sudip Pandey, Xiao M. Chen, M. Brian Maple, Eric E. Fullerton, Sujoy Roy, Claudio Mazzoli, Chandra M. Varma, Sunil K. Sinha

Abstract: In the 1970s a new paradigm was introduced that interacting quenched systems, such as a spin-glass, have a phase transition in which long time memory of spatial patterns is realized without spatial correlations. The principal methods to study the spin-glass transition, besides some elaborate and elegant theoretical constructions, have been numerical computer simulations and neutron spin echo measu… ▽ More In the 1970s a new paradigm was introduced that interacting quenched systems, such as a spin-glass, have a phase transition in which long time memory of spatial patterns is realized without spatial correlations. The principal methods to study the spin-glass transition, besides some elaborate and elegant theoretical constructions, have been numerical computer simulations and neutron spin echo measurements . We show here that the dynamical correlations of the spin-glass transition are embedded in measurements of the four-spin correlations at very long times. This information is directly available in the temporal correlations of the intensity, which encode the spin-orientation memory, obtained by the technique of resonant magnetic x-ray photon correlation spectroscopy (RM- XPCS). We have implemented this method to observe and accurately characterize the critical slowing down of the spin orientation fluctuations in the classic metallic spin glass alloy Cu(Mn) over time scales of 1 to 1000 secs. Our method opens the way for studying phase transitions in systems such as spin ices, and quantum spin liquids, as well as the structural glass transition. △ Less

Submitted 11 February, 2020; originally announced February 2020.

Comments: 14 pages, 9 figures

Ultrafast perturbation of magnetic domains by optical pum** in a ferromagnetic multilayer

Authors: Dmitriy Zusin, Ezio Iacocca, Loïc Le Guyader, Alexander H. Reid, William F. Schlotter, Tian-Min Liu, Daniel J. Higley, Giacomo Coslovich, Scott F. Wandel, Phoebe M. Tengdin, Sheena K. K. Patel, Anatoly Shabalin, Nelson Hua, Stjepan B. Hrkac, Hans T. Nembach, Justin M. Shaw, Sergio A. Montoya, Adam Blonsky, Christian Gentry, Mark A. Hoefer, Margaret M. Murnane, Henry C. Kapteyn, Eric E. Fullerton, Oleg Shpyrko, Hermann A. Dürr , et al. (1 additional authors not shown)

Abstract: Ultrafast optical pum** of spatially nonuniform magnetic textures is known to induce far-from-equilibrium spin transport effects. Here, we use ultrafast x-ray diffraction with unprecedented dynamic range to study the laser-induced dynamics of labyrinth domain networks in ferromagnetic CoFe/Ni multilayers. We detected azimuthally isotropic, odd order, magnetic diffraction rings up to 5th order. T… ▽ More Ultrafast optical pum** of spatially nonuniform magnetic textures is known to induce far-from-equilibrium spin transport effects. Here, we use ultrafast x-ray diffraction with unprecedented dynamic range to study the laser-induced dynamics of labyrinth domain networks in ferromagnetic CoFe/Ni multilayers. We detected azimuthally isotropic, odd order, magnetic diffraction rings up to 5th order. The amplitudes of all three diffraction rings quench to different degrees within 1.6 ps. In addition, all three of the detected diffraction rings both broaden by 15% and radially contract by 6% during the quench process. We are able to rigorously quantify a 31% ultrafast broadening of the domain walls via Fourier analysis of the order-dependent quenching of the three detected diffraction rings. The broadening of the diffraction rings is interpreted as a reduction in the domain coherence length, but the shift in the ring radius, while unambiguous in its occurrence, remains unexplained. In particular, we demonstrate that a radial shift explained by domain wall broadening can be ruled out. With the unprecedented dynamic range of our data, our results provide convincing evidence that labyrinth domain structures are spatially perturbed at ultrafast speeds under far-from-equilibrium conditions, albeit the mechanism inducing the perturbations remains yet to be clarified. △ Less

Submitted 9 June, 2022; v1 submitted 31 January, 2020; originally announced January 2020.

Ultrafast kinetics of the antiferromagnetic-ferromagnetic phase transition in FeRh

Authors: G. Li, R. Medapalli, J. H. Mentink, R. V. Mikhaylovskiy, T. G. H. Blank, S. K. K. Patel, A. K. Zvezdin, Th. Rasing, E. E. Fullerton, A. V. Kimel

Abstract: Understanding how fast short-range interactions build up long-range order is one of the most intriguing topics in condensed matter physics. FeRh is a test specimen for studying this problem in magnetism, where the microscopic spin-spin exchange interaction is ultimately responsible for either ferro- or antiferromagnetic macroscopic order. Femtosecond laser excitation can induce ferromagnetism in a… ▽ More Understanding how fast short-range interactions build up long-range order is one of the most intriguing topics in condensed matter physics. FeRh is a test specimen for studying this problem in magnetism, where the microscopic spin-spin exchange interaction is ultimately responsible for either ferro- or antiferromagnetic macroscopic order. Femtosecond laser excitation can induce ferromagnetism in antiferromagnetic FeRh, but the mechanism and dynamics of this transition are topics of intense debates. Employing double-pump THz emission spectroscopy has enabled us to dramatically increase the temporal detection window of THz emission probes of transient states without sacrificing any loss of resolution or sensitivity. It allows us to study the kinetics of emergent ferromagnetism from the femtosecond up to the nanosecond timescales in FeRh/Pt bilayers. Our results strongly suggest a latency period between the initial pump-excitation and the emission of THz radiation by ferromagnetic nuclei. △ Less

Submitted 21 October, 2021; v1 submitted 19 January, 2020; originally announced January 2020.

Comments: 35 pages total, 8 figures in main text, 7 figures in supplementary

Multiple Kernel Fisher Discriminant Metric Learning for Person Re-identification

Authors: T M Feroz Ali, Kalpesh K Patel, Rajbabu Velmurugan, Subhasis Chaudhuri

Abstract: Person re-identification addresses the problem of matching pedestrian images across disjoint camera views. Design of feature descriptor and distance metric learning are the two fundamental tasks in person re-identification. In this paper, we propose a metric learning framework for person re-identification, where the discriminative metric space is learned using Kernel Fisher Discriminant Analysis (… ▽ More Person re-identification addresses the problem of matching pedestrian images across disjoint camera views. Design of feature descriptor and distance metric learning are the two fundamental tasks in person re-identification. In this paper, we propose a metric learning framework for person re-identification, where the discriminative metric space is learned using Kernel Fisher Discriminant Analysis (KFDA), to simultaneously maximize the inter-class variance as well as minimize the intra-class variance. We derive a Mahalanobis metric induced by KFDA and argue that KFDA is efficient to be applied for metric learning in person re-identification. We also show how the efficiency of KFDA in metric learning can be further enhanced for person re-identification by using two simple yet efficient multiple kernel learning methods. We conduct extensive experiments on three benchmark datasets for person re-identification and demonstrate that the proposed approaches have competitive performance with state-of-the-art methods. △ Less

Submitted 9 October, 2019; originally announced October 2019.

Communication trade-offs for synchronized distributed SGD with large step size

Authors: Kumar Kshitij Patel, Aymeric Dieuleveut

Abstract: Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the \emph{`local-SGD'} model in which the workers train their model independently and synchronize every once in a while. This algorithm improves the… ▽ More Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the \emph{`local-SGD'} model in which the workers train their model independently and synchronize every once in a while. This algorithm improves the computation-communication trade-off but its convergence is not understood very well. We propose a non-asymptotic error analysis, which enables comparison to \emph{one-shot averaging} i.e., a single communication round among independent workers, and \emph{mini-batch averaging} i.e., communicating at every step. We also provide adaptive lower bounds on the communication frequency for large step-sizes ($ t^{-α} $, $ α\in (1/2 , 1 ) $) and show that \emph{Local-SGD} reduces communication by a factor of $O\Big(\frac{\sqrt{T}}{P^{3/2}}\Big)$, with $T$ the total number of gradients and $P$ machines. △ Less

Submitted 25 April, 2019; originally announced April 2019.

Don't Use Large Mini-Batches, Use Local SGD

Authors: Tao Lin, Sebastian U. Stich, Kumar Kshitij Patel, Martin Jaggi

Abstract: Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key efficiency and scalability gains in recent years. However, progress faces a major roadblock, as models trained with large batches often do not generalize well, i.e. they do not show good accuracy on new data. As a remedy, we… ▽ More Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key efficiency and scalability gains in recent years. However, progress faces a major roadblock, as models trained with large batches often do not generalize well, i.e. they do not show good accuracy on new data. As a remedy, we propose a \emph{post-local} SGD and show that it significantly improves the generalization performance compared to large-batch training on standard benchmarks while enjoying the same efficiency (time-to-accuracy) and scalability. We further provide an extensive study of the communication efficiency vs. performance trade-offs associated with a host of \emph{local SGD} variants. △ Less

Submitted 17 February, 2020; v1 submitted 22 August, 2018; originally announced August 2018.

Comments: To appear in ICLR 2020

Laser induced phase transition in epitaxial FeRh layers studied by pump-probe valence band photoemission

Authors: Federico Pressacco, Vojtěch Uhlíř, Matteo Gatti, Alessandro Nicolaou, Azzedine Bendounan, Jon Ander Arregi, Sheena K. K. Patel, Eric E. Fullerton, Damjan Krizmancic, Fausto Sirotti

Abstract: We use time-resolved X-ray photoelectron spectroscopy to probe the electronic and magnetization dynamics in FeRh films after ultrafast laser excitations. We present experimental and theoretical results which investigate the electronic structure of the FeRh during the first-order phase transition identifying a clear signature of the magnetic phase. We find that a spin polarized feature at the Fermi… ▽ More We use time-resolved X-ray photoelectron spectroscopy to probe the electronic and magnetization dynamics in FeRh films after ultrafast laser excitations. We present experimental and theoretical results which investigate the electronic structure of the FeRh during the first-order phase transition identifying a clear signature of the magnetic phase. We find that a spin polarized feature at the Fermi edge is a fingerprint of the magnetic status of the system that is independent of the long-range ferromagnetic alignment of the magnetic domains. We use this feature to follow the phase transition induced by a laser pulse in a pump-probe experiment and find that the magnetic transition occurs in less than 50 ps, and reaches its maximum in 100 ps. △ Less

Submitted 2 March, 2018; originally announced March 2018.

Comments: 16 pages, 5 figures

Photoinduced Enhancement of the Charge Density Wave Amplitude

Authors: A. Singer, S. K. K. Patel, R. Kukreja, V. Uhlíř, J. Wingert, S. Festersen, D. Zhu, J. M. Glownia, H. Lemke, S. Nelson, M. Kozina, K. Rossnagel, M. Bauer, B. M. Murphy, O. M. Magnussen, E. E. Fullerton, O. G. Shpyrko

Abstract: Symmetry breaking and the emergence of order is one of the most fascinating phenomena in condensed matter physics. It leads to a plethora of intriguing ground states found in antiferromagnets, Mott insulators, superconductors, and density-wave systems. Exploiting states of matter far from equilibrium can provide even more striking routes to symmetry-lowered, ordered states. Here, we demonstrate fo… ▽ More Symmetry breaking and the emergence of order is one of the most fascinating phenomena in condensed matter physics. It leads to a plethora of intriguing ground states found in antiferromagnets, Mott insulators, superconductors, and density-wave systems. Exploiting states of matter far from equilibrium can provide even more striking routes to symmetry-lowered, ordered states. Here, we demonstrate for the case of elemental chromium that moderate ultrafast photo-excitation can transiently enhance the charge-density-wave (CDW) amplitude by up to 30% above its equilibrium value, while strong excitations lead to an oscillating, large-amplitude CDW state that persists above the equilibrium transition temperature. Both effects result from dynamic electron-phonon interactions, providing an efficient mechanism to selectively transform a broad excitation of the electronic order into a well defined, long-lived coherent lattice vibration. This mechanism may be exploited to transiently enhance order parameters in other systems with coupled degrees of freedom. △ Less

Submitted 8 February, 2017; v1 submitted 25 November, 2015; originally announced November 2015.

Journal ref: Phys. Rev. Lett. 117, 056401 (2016)