-
CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning
Authors:
Reent Schlegel,
Siddhartha Kumar,
Eirik Rosnes,
Alexandre Graell i Amat
Abstract:
We present two novel federated learning (FL) schemes that mitigate the effect of straggling devices by introducing redundancy on the devices' data across the network. Compared to other schemes in the literature, which deal with stragglers or device dropouts by ignoring their contribution, the proposed schemes do not suffer from the client drift problem. The first scheme, CodedPaddedFL, mitigates t…
▽ More
We present two novel federated learning (FL) schemes that mitigate the effect of straggling devices by introducing redundancy on the devices' data across the network. Compared to other schemes in the literature, which deal with stragglers or device dropouts by ignoring their contribution, the proposed schemes do not suffer from the client drift problem. The first scheme, CodedPaddedFL, mitigates the effect of stragglers while retaining the privacy level of conventional FL. It combines one-time padding for user data privacy with gradient codes to yield straggler resiliency. The second scheme, CodedSecAgg, provides straggler resiliency and robustness against model inversion attacks and is based on Shamir's secret sharing. We apply CodedPaddedFL and CodedSecAgg to a classification problem. For a scenario with 120 devices, CodedPaddedFL achieves a speed-up factor of 18 for an accuracy of 95% on the MNIST dataset compared to conventional FL. Furthermore, it yields similar performance in terms of latency compared to a recently proposed scheme by Prakash et al. without the shortcoming of additional leakage of private data. CodedSecAgg outperforms the state-of-the-art secure aggregation scheme LightSecAgg by a speed-up factor of 6.6-18.7 for the MNIST dataset for an accuracy of 95%.
△ Less
Submitted 3 June, 2022; v1 submitted 16 December, 2021;
originally announced December 2021.
-
Privacy-Preserving Coded Mobile Edge Computing for Low-Latency Distributed Inference
Authors:
Reent Schlegel,
Siddhartha Kumar,
Eirik Rosnes,
Alexandre Graell i Amat
Abstract:
We consider a mobile edge computing scenario where a number of devices want to perform a linear inference $\boldsymbol{W}\boldsymbol{x}$ on some local data $\boldsymbol{x}$ given a network-side matrix $\boldsymbol{W}$. The computation is performed at the network edge over a number of edge servers. We propose a coding scheme that provides information-theoretic privacy against $z$ colluding (honest-…
▽ More
We consider a mobile edge computing scenario where a number of devices want to perform a linear inference $\boldsymbol{W}\boldsymbol{x}$ on some local data $\boldsymbol{x}$ given a network-side matrix $\boldsymbol{W}$. The computation is performed at the network edge over a number of edge servers. We propose a coding scheme that provides information-theoretic privacy against $z$ colluding (honest-but-curious) edge servers, while minimizing the overall latency\textemdash comprising upload, computation, download, and decoding latency\textemdash in the presence of straggling servers. The proposed scheme exploits Shamir's secret sharing to yield data privacy and straggler mitigation, combined with replication to provide spatial diversity for the download. We also propose two variants of the scheme that further reduce latency. For a considered scenario with $9$ edge servers, the proposed scheme reduces the latency by $8\%$ compared to the nonprivate scheme recently introduced by Zhang and Simeone, while providing privacy against an honest-but-curious edge server.
△ Less
Submitted 15 February, 2022; v1 submitted 7 October, 2021;
originally announced October 2021.
-
Coding for Straggler Mitigation in Federated Learning
Authors:
Siddhartha Kumar,
Reent Schlegel,
Eirik Rosnes,
Alexandre Graell i Amat
Abstract:
We present a novel coded federated learning (FL) scheme for linear regression that mitigates the effect of straggling devices while retaining the privacy level of conventional FL. The proposed scheme combines one-time padding to preserve privacy and gradient codes to yield resiliency against stragglers and consists of two phases. In the first phase, the devices share a one-time padded version of t…
▽ More
We present a novel coded federated learning (FL) scheme for linear regression that mitigates the effect of straggling devices while retaining the privacy level of conventional FL. The proposed scheme combines one-time padding to preserve privacy and gradient codes to yield resiliency against stragglers and consists of two phases. In the first phase, the devices share a one-time padded version of their local data with a subset of other devices. In the second phase, the devices and the central server collaboratively and iteratively train a global linear model using gradient codes on the one-time padded local data. To apply one-time padding to real data, our scheme exploits a fixed-point arithmetic representation of the data. Unlike the coded FL scheme recently introduced by Prakash \emph{et al.}, the proposed scheme maintains the same level of privacy as conventional FL while achieving a similar training time. Compared to conventional FL, we show that the proposed scheme achieves a training speed-up factor of $6.6$ and $9.2$ on the MNIST and Fashion-MNIST datasets for an accuracy of $95\%$ and $85\%$, respectively.
△ Less
Submitted 15 February, 2022; v1 submitted 30 September, 2021;
originally announced September 2021.
-
Private Edge Computing for Linear Inference Based on Secret Sharing
Authors:
Reent Schlegel,
Siddhartha Kumar,
Eirik Rosnes,
Alexandre Graell i Amat
Abstract:
We consider an edge computing scenario where users want to perform a linear computation on local, private data and a network-wide, public matrix. Users offload computations to edge servers located at the edge of the network, but do not want the servers, or any other party with access to the wireless links, to gain any information about their data. We provide a scheme that guarantees information-th…
▽ More
We consider an edge computing scenario where users want to perform a linear computation on local, private data and a network-wide, public matrix. Users offload computations to edge servers located at the edge of the network, but do not want the servers, or any other party with access to the wireless links, to gain any information about their data. We provide a scheme that guarantees information-theoretic user data privacy against an eavesdropper with access to a number of edge servers or their corresponding communication links. The novelty of the proposed scheme lies in the utilization of secret sharing and partial replication to provide privacy, mitigate the effect of straggling servers, and to allow for joint beamforming opportunities in the download phase, to minimize the overall latency, consisting of upload, computation, and download latencies.
△ Less
Submitted 19 October, 2020; v1 submitted 18 May, 2020;
originally announced May 2020.
-
ReDS: A Framework for Reputation-Enhanced DHTs
Authors:
Ruj Akavipat,
Mahdi N. Al-Ameen,
Apu Kapadia,
Zahid Rahman,
Roman Schlegel,
Matthew Wright
Abstract:
Distributed Hash Tables (DHTs) such as Chord and Kademlia offer an efficient solution for locating resources in peer-to-peer networks. Unfortunately, malicious nodes along a lookup path can easily subvert such queries. Several systems, including Halo (based on Chord) and Kad (based on Kademlia), mitigate such attacks by using a combination of redundancy and diversity in the paths taken by redundan…
▽ More
Distributed Hash Tables (DHTs) such as Chord and Kademlia offer an efficient solution for locating resources in peer-to-peer networks. Unfortunately, malicious nodes along a lookup path can easily subvert such queries. Several systems, including Halo (based on Chord) and Kad (based on Kademlia), mitigate such attacks by using a combination of redundancy and diversity in the paths taken by redundant lookup queries. Much greater assurance can be provided, however. We describe Reputation for Directory Services (ReDS), a framework for enhancing lookups in redundant DHTs by tracking how well other nodes service lookup requests. We describe how the ReDS technique can be applied to virtually any redundant DHT including Halo and Kad. We also study the collaborative identification and removal of bad lookup paths in a way that does not rely on the sharing of reputation scores --- we show that such sharing is vulnerable to attacks that make it unsuitable for most applications of ReDS. Through extensive simulations we demonstrate that ReDS improves lookup success rates for Halo and Kad by 80% or more over a wide range of conditions, even against strategic attackers attempting to game their reputation scores and in the presence of node churn.
△ Less
Submitted 21 September, 2012;
originally announced September 2012.
