Showing 1–8 of 8 results for author: Huba, D

Confidential Federated Computations

Authors: Hubert Eichner, Daniel Ramage, Kallista Bonawitz, Dzmitry Huba, Tiziano Santoro, Brett McLarnon, Timon Van Overveldt, Nova Fallen, Peter Kairouz, Albert Cheu, Katharine Daly, Adria Gascon, Marco Gruteser, Brendan McMahan

Abstract: Federated Learning and Analytics (FLA) have seen widespread adoption by technology platforms for processing sensitive on-device data. However, basic FLA systems have privacy limitations: they do not necessarily require anonymization mechanisms like differential privacy (DP), and provide limited protections against a potentially malicious service provider. Adding DP to a basic FLA system currently… ▽ More Federated Learning and Analytics (FLA) have seen widespread adoption by technology platforms for processing sensitive on-device data. However, basic FLA systems have privacy limitations: they do not necessarily require anonymization mechanisms like differential privacy (DP), and provide limited protections against a potentially malicious service provider. Adding DP to a basic FLA system currently requires either adding excessive noise to each device's updates, or assuming an honest service provider that correctly implements the mechanism and only uses the privatized outputs. Secure multiparty computation (SMPC) -based oblivious aggregations can limit the service provider's access to individual user updates and improve DP tradeoffs, but the tradeoffs are still suboptimal, and they suffer from scalability challenges and susceptibility to Sybil attacks. This paper introduces a novel system architecture that leverages trusted execution environments (TEEs) and open-sourcing to both ensure confidentiality of server-side computations and provide externally verifiable privacy properties, bolstering the robustness and trustworthiness of private federated computations. △ Less

Submitted 16 April, 2024; originally announced April 2024.

FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learning

Authors: Meisam Hejazinia, Dzmitry Huba, Ilias Leontiadis, Kiwan Maeng, Mani Malek, Luca Melis, Ilya Mironov, Milad Nasr, Kaikai Wang, Carole-Jean Wu

Abstract: Federated learning (FL) has emerged as an effective approach to address consumer privacy needs. FL has been successfully applied to certain machine learning tasks, such as training smart keyboard models and keyword spotting. Despite FL's initial success, many important deep learning use cases, such as ranking and recommendation tasks, have been limited from on-device learning. One of the key chall… ▽ More Federated learning (FL) has emerged as an effective approach to address consumer privacy needs. FL has been successfully applied to certain machine learning tasks, such as training smart keyboard models and keyword spotting. Despite FL's initial success, many important deep learning use cases, such as ranking and recommendation tasks, have been limited from on-device learning. One of the key challenges faced by practical FL adoption for DL-based ranking and recommendation is the prohibitive resource requirements that cannot be satisfied by modern mobile systems. We propose Federated Ensemble Learning (FEL) as a solution to tackle the large memory requirement of deep learning ranking and recommendation tasks. FEL enables large-scale ranking and recommendation model training on-device by simultaneously training multiple model versions on disjoint clusters of client devices. FEL integrates the trained sub-models via an over-arch layer into an ensemble model that is hosted on the server. Our experiments demonstrate that FEL leads to 0.43-2.31% model quality improvement over traditional on-device federated learning - a significant improvement for ranking and recommendation system use cases. △ Less

Submitted 7 June, 2022; originally announced June 2022.

Papaya: Practical, Private, and Scalable Federated Learning

Authors: Dzmitry Huba, John Nguyen, Kshitiz Malik, Ruiyu Zhu, Mike Rabbat, Ashkan Yousefpour, Carole-Jean Wu, Hongyuan Zhan, Pavel Ustinov, Harish Srinivas, Kaikai Wang, Anthony Shoumikhin, Jesik Min, Mani Malek

Abstract: Cross-device Federated Learning (FL) is a distributed learning paradigm with several challenges that differentiate it from traditional distributed learning, variability in the system characteristics on each device, and millions of clients coordinating with a central server being primary ones. Most FL systems described in the literature are synchronous - they perform a synchronized aggregation of m… ▽ More Cross-device Federated Learning (FL) is a distributed learning paradigm with several challenges that differentiate it from traditional distributed learning, variability in the system characteristics on each device, and millions of clients coordinating with a central server being primary ones. Most FL systems described in the literature are synchronous - they perform a synchronized aggregation of model updates from individual clients. Scaling synchronous FL is challenging since increasing the number of clients training in parallel leads to diminishing returns in training speed, analogous to large-batch training. Moreover, stragglers hinder synchronous FL training. In this work, we outline a production asynchronous FL system design. Our work tackles the aforementioned issues, sketches of some of the system design challenges and their solutions, and touches upon principles that emerged from building a production FL system for millions of clients. Empirically, we demonstrate that asynchronous FL converges faster than synchronous FL when training across nearly one hundred million devices. In particular, in high concurrency settings, asynchronous FL is 5x faster and has nearly 8x less communication overhead than synchronous FL. △ Less

Submitted 25 April, 2022; v1 submitted 8 November, 2021; originally announced November 2021.

Federated Learning with Buffered Asynchronous Aggregation

Authors: John Nguyen, Kshitiz Malik, Hongyuan Zhan, Ashkan Yousefpour, Michael Rabbat, Mani Malek, Dzmitry Huba

Abstract: Scalability and privacy are two critical concerns for cross-device federated learning (FL) systems. In this work, we identify that synchronous FL - synchronized aggregation of client updates in FL - cannot scale efficiently beyond a few hundred clients training in parallel. It leads to diminishing returns in model performance and training speed, analogous to large-batch training. On the other hand… ▽ More Scalability and privacy are two critical concerns for cross-device federated learning (FL) systems. In this work, we identify that synchronous FL - synchronized aggregation of client updates in FL - cannot scale efficiently beyond a few hundred clients training in parallel. It leads to diminishing returns in model performance and training speed, analogous to large-batch training. On the other hand, asynchronous aggregation of client updates in FL (i.e., asynchronous FL) alleviates the scalability issue. However, aggregating individual client updates is incompatible with Secure Aggregation, which could result in an undesirable level of privacy for the system. To address these concerns, we propose a novel buffered asynchronous aggregation method, FedBuff, that is agnostic to the choice of optimizer, and combines the best properties of synchronous and asynchronous FL. We empirically demonstrate that FedBuff is 3.3x more efficient than synchronous FL and up to 2.5x more efficient than asynchronous FL, while being compatible with privacy-preserving technologies such as Secure Aggregation and differential privacy. We provide theoretical convergence guarantees in a smooth non-convex setting. Finally, we show that under differentially private training, FedBuff can outperform FedAvgM at low privacy settings and achieve the same utility for higher privacy settings. △ Less

Submitted 7 March, 2022; v1 submitted 11 June, 2021; originally announced June 2021.

Comments: Accepted at AISTATS 2022. Previously accepted at FL-ICML 2021

Towards Federated Learning at Scale: System Design

Authors: Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, Jason Roselander

Abstract: Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and… ▽ More Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and future directions. △ Less

Submitted 22 March, 2019; v1 submitted 4 February, 2019; originally announced February 2019.

Anthropogenic Space Weather

Authors: T. I. Gombosi, D. N. Baker, A. Balogh, P. J. Erickson, J. D. Huba, L. J. Lanzerotti

Abstract: Anthropogenic effects on the space environment started in the late 19th century and reached their peak in the 1960s when high-altitude nuclear explosions were carried out by the USA and the Soviet Union. These explosions created artificial radiation belts near Earth that resulted in major damages to several satellites. Another, unexpected impact of the high-altitude nuclear tests was the electroma… ▽ More Anthropogenic effects on the space environment started in the late 19th century and reached their peak in the 1960s when high-altitude nuclear explosions were carried out by the USA and the Soviet Union. These explosions created artificial radiation belts near Earth that resulted in major damages to several satellites. Another, unexpected impact of the high-altitude nuclear tests was the electromagnetic pulse (EMP) that can have devastating effects over a large geographic area (as large as the continental United States). Other anthropogenic impacts on the space environment include chemical release ex- periments, high-frequency wave heating of the ionosphere and the interaction of VLF waves with the radiation belts. This paper reviews the fundamental physical process behind these phenomena and discusses the observations of their impacts. △ Less

Submitted 23 March, 2017; v1 submitted 10 November, 2016; originally announced November 2016.

Comments: 71 pages, 35 figures

Ionized Plasma and Neutral Gas Coupling in the Sun's Chromosphere and Earth's Ionosphere/Thermosphere

Authors: J. E. Leake, C. R. DeVore, J. P. Thayer, A. G. Burns, G. Crowley, H. R. Gilbert, J. D Huba, J. Krall, M. G. Linton, V. S. Lukin, W. Wang

Abstract: We review our understanding of ionized plasma and neutral gas coupling in the weakly ionized, stratified, electromagnetically-permeated regions of the Sun's chromosphere and Earth's ionosphere/thermosphere. Using representative models for each environment we derive fundamental descriptions of the coupling of the constituent parts to each other and to the electric and magnetic fields, and we examin… ▽ More We review our understanding of ionized plasma and neutral gas coupling in the weakly ionized, stratified, electromagnetically-permeated regions of the Sun's chromosphere and Earth's ionosphere/thermosphere. Using representative models for each environment we derive fundamental descriptions of the coupling of the constituent parts to each other and to the electric and magnetic fields, and we examine the variation in magnetization of the ionized component. Using these descriptions we compare related phenomena in the two environments, and discuss electric currents, energy transfer and dissipation. We present a coupled theoretical and numerical study of plasma instabilities in the two environments that serves as an example of how the chromospheric and ionospheric communities can further collaborate. We also suggest future collaborative studies that will help improve our understanding of these two different atmospheres which share many similarities, but have large disparities in key quantities. △ Less

Submitted 29 August, 2016; v1 submitted 1 October, 2013; originally announced October 2013.

Simulation Study of a Positive Ionospheric Storm Phase Observed at Millstone Hill

Authors: M. Swisdak, J. D. Huba, G. Joyce, Chao-Song Huang

Abstract: Simulation results from the NRL ionospheric model SAMI2 indicate that the changes in the F-region over Millstone Hill during the geomagnetic storm beginning on 3 April 2004 are primarily due to the influence of a long-lasting eastward electric field, as was previously suggested by {\it Huang et al.} [2005]. A simulation of the storm day agrees well with the observational data and shows that, com… ▽ More Simulation results from the NRL ionospheric model SAMI2 indicate that the changes in the F-region over Millstone Hill during the geomagnetic storm beginning on 3 April 2004 are primarily due to the influence of a long-lasting eastward electric field, as was previously suggested by {\it Huang et al.} [2005]. A simulation of the storm day agrees well with the observational data and shows that, compared with the ionosphere of the previous quiet day, the peak electron density in the F-region (NmF2) increased by a factor of $\approx 2$, the altitude of the peak density (hmF2) rose by $\approx 80$ km, and the F-region electron temperature decreased by $\approx 1000$ K. Further simulations in which either the neutral atmosphere and winds or the electric field were replaced by their quiet day counterparts clearly suggest that the electric field played the dominant, although not exclusive, role in producing these effects. △ Less

Submitted 14 December, 2005; originally announced December 2005.

Comments: Submitted for publication in GRL