Showing 1–50 of 73 results for author: Kliewer, J

VALID: a Validated Algorithm for Learning in Decentralized Networks with Possible Adversarial Presence

Authors: Mayank Bakshi, Sara Ghasvarianjahromi, Yauhen Yakimenka, Allison Beemer, Oliver Kosut, Joerg Kliewer

Abstract: We introduce the paradigm of validated decentralized learning for undirected networks with heterogeneous data and possible adversarial infiltration. We require (a) convergence to a global empirical loss minimizer when adversaries are absent, and (b) either detection of adversarial presence of convergence to an admissible consensus irrespective of the adversarial configuration. To this end, we prop… ▽ More We introduce the paradigm of validated decentralized learning for undirected networks with heterogeneous data and possible adversarial infiltration. We require (a) convergence to a global empirical loss minimizer when adversaries are absent, and (b) either detection of adversarial presence of convergence to an admissible consensus irrespective of the adversarial configuration. To this end, we propose the VALID protocol which, to the best of our knowledge, is the first to achieve a validated learning guarantee. Moreover, VALID offers an O(1/T) convergence rate (under pertinent regularity assumptions), and computational and communication complexities comparable to non-adversarial distributed stochastic gradient descent. Remarkably, VALID retains optimal performance metrics in adversary-free environments, sidestep** the robustness penalties observed in prior byzantine-robust methods. A distinctive aspect of our study is a heterogeneity metric based on the norms of individual agents' gradients computed at the global empirical loss minimizer. This not only provides a natural statistic for detecting significant byzantine disruptions but also allows us to prove the optimality of VALID in wide generality. Lastly, our numerical results reveal that, in the absence of adversaries, VALID converges faster than state-of-the-art byzantine robust algorithms, while when adversaries are present, VALID terminates with each honest either converging to an admissible consensus of declaring adversarial presence in the network. △ Less

Submitted 12 May, 2024; originally announced May 2024.

Comments: This is an extended version of the paper at International Symposium on Information Theory 2024

Secure Distributed Storage: Optimal Trade-Off Between Storage Rate and Privacy Leakage

Authors: Remi A. Chou, Joerg Kliewer

Abstract: Consider the problem of storing data in a distributed manner over $T$ servers. Specifically, the data needs to (i) be recoverable from any $τ$ servers, and (ii) remain private from any $z$ colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at any $z$ colluding servers. For this model, our main results are (i) the fundame… ▽ More Consider the problem of storing data in a distributed manner over $T$ servers. Specifically, the data needs to (i) be recoverable from any $τ$ servers, and (ii) remain private from any $z$ colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at any $z$ colluding servers. For this model, our main results are (i) the fundamental trade-off between storage size and the level of desired privacy, and (ii) the optimal amount of local randomness necessary at the encoder. As a byproduct, our results provide an optimal lower bound on the individual share size of ramp secret sharing schemes under a more general leakage symmetry condition than the ones previously considered in the literature. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Comments: 11 pages, 3 figures, two-column, accepted to IEEE Transactions on Information Theory, part of the results was presented at the 2020 IEEE International Symposium on Information Theory (ISIT)

Uncoded Storage Coded Transmission Elastic Computing with Straggler Tolerance in Heterogeneous Systems

Authors: Xi Zhong, Joerg Kliewer, Mingyue Ji

Abstract: In 2018, Yang et al. introduced a novel and effective approach, using maximum distance separable (MDS) codes, to mitigate the impact of elasticity in cloud computing systems. This approach is referred to as coded elastic computing. Some limitations of this approach include that it assumes all virtual machines have the same computing speeds and storage capacities, and it cannot tolerate stragglers… ▽ More In 2018, Yang et al. introduced a novel and effective approach, using maximum distance separable (MDS) codes, to mitigate the impact of elasticity in cloud computing systems. This approach is referred to as coded elastic computing. Some limitations of this approach include that it assumes all virtual machines have the same computing speeds and storage capacities, and it cannot tolerate stragglers for matrix-matrix multiplications. In order to resolve these limitations, in this paper, we introduce a new combinatorial optimization framework, named uncoded storage coded transmission elastic computing (USCTEC), for heterogeneous speeds and storage constraints, aiming to minimize the expected computation time for matrix-matrix multiplications, under the consideration of straggler tolerance. Within this framework, we propose optimal solutions with straggler tolerance under relaxed storage constraints. Moreover, we propose a heuristic algorithm that considers the heterogeneous storage constraints. Our results demonstrate that the proposed algorithm outperforms baseline solutions utilizing cyclic storage placements, in terms of both expected computation time and storage size. △ Less

Submitted 22 January, 2024; originally announced January 2024.

Comments: 6 pages, 1 figure, accepted in ICC 2024

FedDRO: Federated Compositional Optimization for Distributionally Robust Learning

Authors: Prashant Khanduri, Chengyin Li, Rafi Ibn Sultan, Yao Qiang, Joerg Kliewer, Dongxiao Zhu

Abstract: Recently, compositional optimization (CO) has gained popularity because of its applications in distributionally robust optimization (DRO) and many other machine learning problems. Large-scale and distributed availability of data demands the development of efficient federated learning (FL) algorithms for solving CO problems. Develo** FL algorithms for CO is particularly challenging because of the… ▽ More Recently, compositional optimization (CO) has gained popularity because of its applications in distributionally robust optimization (DRO) and many other machine learning problems. Large-scale and distributed availability of data demands the development of efficient federated learning (FL) algorithms for solving CO problems. Develo** FL algorithms for CO is particularly challenging because of the compositional nature of the objective. Moreover, current state-of-the-art methods to solve such problems rely on large batch gradients (depending on the solution accuracy) not feasible for most practical settings. To address these challenges, in this work, we propose efficient FedAvg-type algorithms for solving non-convex CO in the FL setting. We first establish that vanilla FedAvg is not suitable to solve distributed CO problems because of the data heterogeneity in the compositional objective at each client which leads to the amplification of bias in the local compositional gradient estimates. To this end, we propose a novel FL framework FedDRO that utilizes the DRO problem structure to design a communication strategy that allows FedAvg to control the bias in the estimation of the compositional gradient. A key novelty of our work is to develop solution accuracy-independent algorithms that do not require large batch gradients (and function evaluations) for solving federated CO problems. We establish $\mathcal{O}(ε^{-2})$ sample and $\mathcal{O}(ε^{-3/2})$ communication complexity in the FL setting while achieving linear speedup with the number of clients. We corroborate our theoretical findings with empirical studies on large-scale DRO problems. △ Less

Submitted 21 November, 2023; originally announced November 2023.

Comments: 38 Pages, 6 Figures

On the Capacity of Private Nonlinear Computation for Replicated Databases

Authors: Sarah A. Obead, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

Abstract: We consider the problem of private computation (PC) in a distributed storage system. In such a setting a user wishes to compute a function of $f$ messages replicated across $n$ noncolluding databases, while revealing no information about the desired function to the databases. We provide an information-theoretically accurate achievable PC rate, which is the ratio of the smallest desired amount of i… ▽ More We consider the problem of private computation (PC) in a distributed storage system. In such a setting a user wishes to compute a function of $f$ messages replicated across $n$ noncolluding databases, while revealing no information about the desired function to the databases. We provide an information-theoretically accurate achievable PC rate, which is the ratio of the smallest desired amount of information and the total amount of downloaded information, for the scenario of nonlinear computation. For a large message size the rate equals the PC capacity, i.e., the maximum achievable PC rate, when the candidate functions are the $f$ independent messages and one arbitrary nonlinear function of these. When the number of messages grows, the PC rate approaches an outer bound on the PC capacity. As a special case, we consider private monomial computation (PMC) and numerically compare the achievable PMC rate to the outer bound for a finite number of messages. △ Less

Submitted 4 July, 2023; originally announced July 2023.

Comments: 5 pages, 1 figure, 1 table. Presented at the 2019 IEEE Information Theory Workshop (ITW). Figure 1 is updated as it contained incorrect data-points for $f=2$ and $g=3$. arXiv admin note: text overlap with arXiv:2003.10007

Journal ref: in Proc. IEEE Inf. Theory Workshop (ITW), Visby, Sweden, Aug. 25-28, 2019, pp. 1-5

Straggler-Resilient Differentially-Private Decentralized Learning

Authors: Yauhen Yakimenka, Chung-Wei Weng, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

Abstract: We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency--comprising both computation and communication latency. Analytical results on both the convergence speed and… ▽ More We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency--comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for both a skip** scheme (which ignores the stragglers after a timeout) and a baseline scheme that waits for each node to finish before the training continues. A trade-off between overall training latency, accuracy, and privacy, parameterized by the timeout of the skip** scheme, is identified and empirically validated for logistic regression on a real-world dataset and for image classification using the MNIST and CIFAR-10 datasets. △ Less

Submitted 28 June, 2024; v1 submitted 6 December, 2022; originally announced December 2022.

Comments: To appear in the IEEE Journal on Selected Areas in Information Theory (special issue on Information-Theoretic Methods for Trustworthy and Reliable Machine Learning)

Pliable Private Information Retrieval

Authors: Sarah A. Obead, Jörg Kliewer

Abstract: We formulate a new variant of the private information retrieval (PIR) problem where the user is pliable, i.e., interested in any message from a desired subset of the available dataset, denoted as pliable private information retrieval (PPIR). We consider a setup where a dataset consisting of $f$ messages is replicated in $n$ noncolluding databases and classified into $Γ$ classes. For this setup, th… ▽ More We formulate a new variant of the private information retrieval (PIR) problem where the user is pliable, i.e., interested in any message from a desired subset of the available dataset, denoted as pliable private information retrieval (PPIR). We consider a setup where a dataset consisting of $f$ messages is replicated in $n$ noncolluding databases and classified into $Γ$ classes. For this setup, the user wishes to retrieve any $λ\geq 1$ messages from multiple desired classes, i.e., $η\geq 1$, while revealing no information about the identity of the desired classes to the databases. We term this problem multi-message PPIR (M-PPIR) and introduce the single-message PPIR (PPIR) problem as an elementary special case of M-PPIR. We first derive converse bounds on the M-PPIR rate, which is defined as the ratio of the desired amount of information and the total amount of downloaded information, followed by the corresponding achievable schemes. As a result, we show that the PPIR capacity, i.e., the maximum achievable PPIR rate, for $n$ noncolluding databases matches the capacity of PIR with $n$ databases and $Γ$ messages. Thus, enabling flexibility, i.e., pliability, where privacy is only guaranteed for classes, but not for messages as in classical PIR, allows to trade-off privacy versus download rate. A similar insight is shown to hold for the general case of M-PPIR. △ Less

Submitted 12 June, 2022; originally announced June 2022.

Comments: 23 pages, 3 figures, 3 tables, submitted for possible publication

Information Leakage in Index Coding With Sensitive and Non-Sensitive Messages

Authors: Yucheng Liu, Lawrence Ong, Phee Lep Yeoh, Parastoo Sadeghi, Joerg Kliewer, Sarah Johnson

Abstract: Information leakage to a guessing adversary in index coding is studied, where some messages in the system are sensitive and others are not. The non-sensitive messages can be used by the server like secret keys to mitigate leakage of the sensitive messages to the adversary. We construct a deterministic linear coding scheme, developed from the rank minimization method based on fitting matrices (Bar-… ▽ More Information leakage to a guessing adversary in index coding is studied, where some messages in the system are sensitive and others are not. The non-sensitive messages can be used by the server like secret keys to mitigate leakage of the sensitive messages to the adversary. We construct a deterministic linear coding scheme, developed from the rank minimization method based on fitting matrices (Bar-Yossef et al. 2011). The linear scheme leads to a novel upper bound on the optimal information leakage rate, which is proved to be tight over all deterministic scalar linear codes. We also derive a converse result from a graph-theoretic perspective, which holds in general over all deterministic and stochastic coding schemes. △ Less

Submitted 22 May, 2022; originally announced May 2022.

Comments: Accepted by IEEE International Symposium on Information Theory (ISIT) 2022

Information Leakage in Index Coding

Authors: Yucheng Liu, Lawrence Ong, Phee Lep Yeoh, Parastoo Sadeghi, Joerg Kliewer, Sarah Johnson

Abstract: We study the information leakage to a guessing adversary in index coding with a general message distribution. Under both vanishing-error and zero-error decoding assumptions, we develop lower and upper bounds on the optimal leakage rate, which are based on the broadcast rate of the subproblem induced by the set of messages the adversary tries to guess. When the messages are independent and uniforml… ▽ More We study the information leakage to a guessing adversary in index coding with a general message distribution. Under both vanishing-error and zero-error decoding assumptions, we develop lower and upper bounds on the optimal leakage rate, which are based on the broadcast rate of the subproblem induced by the set of messages the adversary tries to guess. When the messages are independent and uniformly distributed, the lower and upper bounds match, establishing an equivalence between the two rates. △ Less

Submitted 22 May, 2022; originally announced May 2022.

Comments: Published in Proceedings of IEEE Information Theory Workshop (ITW) 2021

Function Computation Without Secure Links: Information and Leakage Rates

Authors: Remi A. Chou, Joerg Kliewer

Abstract: Consider L users, who each holds private data, and one fusion center who must compute a function of the private data of the L users. To accomplish this task, each user can make a single use of a public and noiseless broadcast channel. In this setting, and in the absence of any additional resources such as secure links, we study the optimal communication rates and minimum information leakages on th… ▽ More Consider L users, who each holds private data, and one fusion center who must compute a function of the private data of the L users. To accomplish this task, each user can make a single use of a public and noiseless broadcast channel. In this setting, and in the absence of any additional resources such as secure links, we study the optimal communication rates and minimum information leakages on the private user data that are achievable. Specifically, we study the information leakage of the user data at the fusion center (beyond the knowledge of the function output), as well as at predefined groups of colluding users who eavesdrop one another. We derive the capacity region when the user data is independent, and inner and outer regions for the capacity region when the user data is correlated. △ Less

Submitted 27 January, 2022; originally announced January 2022.

Comments: Submitted to the 2022 IEEE International Symposium on Information Theory

RELDEC: Reinforcement Learning-Based Decoding of Moderate Length LDPC Codes

Authors: Salman Habib, Allison Beemer, Joerg Kliewer

Abstract: In this work we propose RELDEC, a novel approach for sequential decoding of moderate length low-density parity-check (LDPC) codes. The main idea behind RELDEC is that an optimized decoding policy is subsequently obtained via reinforcement learning based on a Markov decision process (MDP). In contrast to our previous work, where an agent learns to schedule only a single check node (CN) within a gro… ▽ More In this work we propose RELDEC, a novel approach for sequential decoding of moderate length low-density parity-check (LDPC) codes. The main idea behind RELDEC is that an optimized decoding policy is subsequently obtained via reinforcement learning based on a Markov decision process (MDP). In contrast to our previous work, where an agent learns to schedule only a single check node (CN) within a group (cluster) of CNs per iteration, in this work we train the agent to schedule all CNs in a cluster, and all clusters in every iteration. That is, in each learning step of RELDEC an agent learns to schedule CN clusters sequentially depending on a reward associated with the outcome of scheduling a particular cluster. We also modify the state space representation of the MDP, enabling RELDEC to be suitable for larger block length LDPC codes than those studied in our previous work. Furthermore, to address decoding under varying channel conditions, we propose agile meta-RELDEC (AM-RELDEC) that employs meta-reinforcement learning. The proposed RELDEC scheme significantly outperforms standard flooding and random sequential decoding for a variety of LDPC codes, including codes designed for 5G new radio. △ Less

Submitted 27 July, 2023; v1 submitted 27 December, 2021; originally announced December 2021.

Comments: Accepted for publication in IEEE Transactions on Communications

Keyless Authentication for AWGN Channels

Authors: Eric Graves, Allison Beemer, Jorg Kliewer, Oliver Kosut, Paul Yu

Abstract: This work establishes that the physical layer can be used to perform information-theoretic authentication in additive white Gaussian noise channels, as long as the adversary is not omniscient. The model considered consists of an encoder, decoder, and adversary, where the adversary has access to the message, a non-causal noisy observation of the encoder's transmission, and unlimited transmission po… ▽ More This work establishes that the physical layer can be used to perform information-theoretic authentication in additive white Gaussian noise channels, as long as the adversary is not omniscient. The model considered consists of an encoder, decoder, and adversary, where the adversary has access to the message, a non-causal noisy observation of the encoder's transmission, and unlimited transmission power, while the decoder observes a noisy version of the sum of the encoder and adversary's outputs. A method to modify a generic existing channel code to enable authentication is presented. It is shown that this modification costs an asymptotically negligible amount of the coding rate, while still enabling authentication as long as the adversary's observation is not noiseless. Also notable is that this modification is not (asymptotically) a function of the statistical characterization of the adversary's channel and furthermore no secret key is required, hence paving the way for a robust practical implementation. Using these results, the channel-authenticated capacity is calculated and shown to be equal to the non-adversarial channel capacity. While this modular scheme is designed for use in the given channel model, it is applicable to a wide range of settings. △ Less

Submitted 11 November, 2021; originally announced November 2021.

Comments: Submitted for possible publication

Optimal Rate-Distortion-Leakage Tradeoff for Single-Server Information Retrieval

Authors: Yauhen Yakimenka, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

Abstract: Private information retrieval protocols guarantee that a user can privately and losslessly retrieve a single file from a database stored across multiple servers. In this work, we propose to simultaneously relax the conditions of perfect retrievability and privacy in order to obtain improved download rates when all files are stored uncoded on a single server. Information leakage is measured in term… ▽ More Private information retrieval protocols guarantee that a user can privately and losslessly retrieve a single file from a database stored across multiple servers. In this work, we propose to simultaneously relax the conditions of perfect retrievability and privacy in order to obtain improved download rates when all files are stored uncoded on a single server. Information leakage is measured in terms of the average success probability for the server of correctly guessing the identity of the desired file. The main findings are: i) The derivation of the optimal tradeoff between download rate, distortion, and information leakage when the file size is infinite. Closed-form expressions of the optimal tradeoff for the special cases of "no-leakage" and "no-privacy" are also given. ii) A novel approach based on linear programming (LP) to construct schemes for a finite file size and an arbitrary number of files. The proposed LP approach can be leveraged to find provably optimal schemes with corresponding closed-form expressions for the rate-distortion-leakage tradeoff when the database contains at most four bits. Finally, for a database that contains 320 bits, we compare two construction methods based on the LP approach with a nonconstructive scheme downloading subsets of files using a finite-length lossy compressor based on random coding. △ Less

Submitted 6 January, 2022; v1 submitted 9 November, 2021; originally announced November 2021.

Comments: 14 pages, 3 figures. Accepted for publication in IEEE Journal on Selected Areas in Communications, Special Issue on Private Information Retrieval, Private Coded Computing over Distributed Servers, and Privacy in Distributed Learning

A Practical Algorithm Design and Evaluation for Heterogeneous Elastic Computing with Stragglers

Authors: Nicholas Woolsey, Joerg Kliewer, Rong-Rong Chen, Mingyue Ji

Abstract: Our extensive real measurements over Amazon EC2 show that the virtual instances often have different computing speeds even if they share the same configurations. This motivates us to study heterogeneous Coded Storage Elastic Computing (CSEC) systems where machines, with different computing speeds, join and leave the network arbitrarily over different computing steps. In CSEC systems, a Maximum Dis… ▽ More Our extensive real measurements over Amazon EC2 show that the virtual instances often have different computing speeds even if they share the same configurations. This motivates us to study heterogeneous Coded Storage Elastic Computing (CSEC) systems where machines, with different computing speeds, join and leave the network arbitrarily over different computing steps. In CSEC systems, a Maximum Distance Separable (MDS) code is used for coded storage such that the file placement does not have to be redefined with each elastic event. Computation assignment algorithms are used to minimize the computation time given computation speeds of different machines. While previous studies of heterogeneous CSEC do not include stragglers-the slow machines during the computation, we develop a new framework in heterogeneous CSEC that introduces straggler tolerance. Based on this framework, we design a novel algorithm using our previously proposed approach for heterogeneous CSEC such that the system can handle any subset of stragglers of a specified size while minimizing the computation time. Furthermore, we establish a trade-off in computation time and straggler tolerance. Another major limitation of existing CSEC designs is the lack of practical evaluations using real applications. In this paper, we evaluate the performance of our designs on Amazon EC2 for applications of the power iteration and linear regression. Evaluation results show that the proposed heterogeneous CSEC algorithms outperform the state-of-the-art designs by more than 30%. △ Less

Submitted 15 September, 2021; v1 submitted 18 July, 2021; originally announced July 2021.

Comments: 6 pages, 2 figures, accepted by IEEE Globecom 2021

Doubly-Exponential Identification via Channels: Code Constructions and Bounds

Authors: Onur Günlü, Joerg Kliewer, Rafael F. Schaefer, Vladimir Sidorenko

Abstract: Consider the identification (ID) via channels problem, where a receiver wants to decide whether the transmitted identifier is its identifier, rather than decoding the identifier. This model allows to transmit identifiers whose size scales doubly-exponentially in the blocklength, unlike common transmission (or channel) codes whose size scales exponentially. It suffices to use binary constant-weight… ▽ More Consider the identification (ID) via channels problem, where a receiver wants to decide whether the transmitted identifier is its identifier, rather than decoding the identifier. This model allows to transmit identifiers whose size scales doubly-exponentially in the blocklength, unlike common transmission (or channel) codes whose size scales exponentially. It suffices to use binary constant-weight codes (CWCs) to achieve the ID capacity. By relating the parameters of a binary CWC to the minimum distance of a code and using higher-order correlation moments, two upper bounds on the binary CWC size are proposed. These bounds are shown to be upper bounds also on the identifier sizes for ID codes constructed by using binary CWCs. We propose two code constructions based on optical orthogonal codes, which are used in optical multiple access schemes, have constant-weight codewords, and satisfy cyclic cross-correlation and auto-correlation constraints. These constructions are modified and concatenated with outer Reed-Solomon codes to propose new binary CWCs optimal for ID. Improvements to the finite-parameter performance of both our and existing code constructions are shown by using outer codes with larger minimum distance vs. blocklength ratios. We also illustrate ID performance regimes for which our ID code constructions perform significantly better than existing constructions. △ Less

Submitted 25 June, 2021; originally announced June 2021.

Comments: To appear in the IEEE International Symposium on Information Theory 2021

Nested Array-Based Spatially Coupled LDPC Codes

Authors: Salman Habib, David G. M. Mitchell, Joerg Kliewer

Abstract: Linear nested codes, where two or more sub-codes are nested in a global code, have been proposed as candidates for reliable multi-terminal communication. In this paper, we consider nested array-based spatially coupled low-density parity-check (SC-LDPC) codes and propose a line-counting based optimization scheme for minimizing the number of dominant absorbing sets in order to improve its performanc… ▽ More Linear nested codes, where two or more sub-codes are nested in a global code, have been proposed as candidates for reliable multi-terminal communication. In this paper, we consider nested array-based spatially coupled low-density parity-check (SC-LDPC) codes and propose a line-counting based optimization scheme for minimizing the number of dominant absorbing sets in order to improve its performance in the high signal-to-noise ratio regime. Since the parity-check matrices of different nested sub-codes partially overlap, the optimization of one nested sub-code imposes constraints on the optimization of the other sub-codes. To tackle these constraints, a multi-step optimization process is applied first to one of the nested codes, then sequential optimization of the remaining nested codes is carried out based on the constraints imposed by the previously optimized sub-codes. Results show that the order of optimization has a significant impact on the number of dominant absorbing sets in the Tanner graph of the code, resulting in a tradeoff between the performance of a nested code structure and its optimization sequence: the code which is optimized without constraints has fewer harmful structures than the code which is optimized with constraints. We also show that for certain code parameters, dominant absorbing sets in the Tanner graphs of all nested codes are completely removed using our proposed optimization strategy. △ Less

Submitted 21 February, 2021; originally announced February 2021.

Comments: Accepted for publication in IEEE Transactions on Communications

Information Leakage in Zero-Error Source Coding: A Graph-Theoretic Perspective

Authors: Yucheng Liu, Lawrence Ong, Sarah Johnson, Joerg Kliewer, Parastoo Sadeghi, Phee Lep Yeoh

Abstract: We study the information leakage to a guessing adversary in zero-error source coding. The source coding problem is defined by a confusion graph capturing the distinguishability between source symbols. The information leakage is measured by the ratio of the adversary's successful guessing probability after and before eavesdrop** the codeword, maximized over all possible source distributions. Such… ▽ More We study the information leakage to a guessing adversary in zero-error source coding. The source coding problem is defined by a confusion graph capturing the distinguishability between source symbols. The information leakage is measured by the ratio of the adversary's successful guessing probability after and before eavesdrop** the codeword, maximized over all possible source distributions. Such measurement under the basic adversarial model where the adversary makes a single guess and allows no distortion between its estimator and the true sequence is known as the maximum min-entropy leakage or the maximal leakage in the literature. We develop a single-letter characterization of the optimal normalized leakage under the basic adversarial model, together with an optimum-achieving scalar stochastic map** scheme. An interesting observation is that the optimal normalized leakage is equal to the optimal compression rate with fixed-length source codes, both of which can be simultaneously achieved by some deterministic coding schemes. We then extend the leakage measurement to generalized adversarial models where the adversary makes multiple guesses and allows certain level of distortion, for which we derive single-letter lower and upper bounds. △ Less

Submitted 3 February, 2021; originally announced February 2021.

Comments: A shortened version has been submitted to ISIT 2021

Generative Adversarial User Privacy in Lossy Single-Server Information Retrieval

Authors: Chung-Wei Weng, Yauhen Yakimenka, Hsuan-Yin Lin, Eirik Rosnes, Joerg Kliewer

Abstract: We propose to extend the concept of private information retrieval by allowing for distortion in the retrieval process and relaxing the perfect privacy requirement at the same time. In particular, we study the trade-off between download rate, distortion, and user privacy leakage, and show that in the limit of large file sizes this trade-off can be captured via a novel information-theoretical formul… ▽ More We propose to extend the concept of private information retrieval by allowing for distortion in the retrieval process and relaxing the perfect privacy requirement at the same time. In particular, we study the trade-off between download rate, distortion, and user privacy leakage, and show that in the limit of large file sizes this trade-off can be captured via a novel information-theoretical formulation for datasets with a known distribution. Moreover, for scenarios where the statistics of the dataset is unknown, we propose a new deep learning framework by leveraging a generative adversarial network approach, which allows the user to learn efficient schemes from the data itself. We evaluate the performance of the scheme on a synthetic Gaussian dataset as well as on the MNIST, CIFAR-10, and LSUN datasets. For the MNIST, CIFAR-10, and LSUN datasets, the data-driven approach significantly outperforms a nonlearning-based scheme which combines source coding with the download of multiple files. △ Less

Submitted 19 October, 2022; v1 submitted 7 December, 2020; originally announced December 2020.

Comments: Accepted for Publication in IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY (TIFS)

Journal ref: IEEE Transactions on Information Forensics and Security, vol. 17, pp. 3495-3510, 2022

Learning to Decode: Reinforcement Learning for Decoding of Sparse Graph-Based Channel Codes

Authors: Salman Habib, Allison Beemer, Joerg Kliewer

Abstract: We show in this work that reinforcement learning can be successfully applied to decoding short to moderate length sparse graph-based channel codes. Specifically, we focus on low-density parity check (LDPC) codes, which for example have been standardized in the context of 5G cellular communication systems due to their excellent error correcting performance. These codes are typically decoded via bel… ▽ More We show in this work that reinforcement learning can be successfully applied to decoding short to moderate length sparse graph-based channel codes. Specifically, we focus on low-density parity check (LDPC) codes, which for example have been standardized in the context of 5G cellular communication systems due to their excellent error correcting performance. These codes are typically decoded via belief propagation iterative decoding on the corresponding bipartite (Tanner) graph of the code via flooding, i.e., all check and variable nodes in the Tanner graph are updated at once. In contrast, in this paper we utilize a sequential update policy which selects the optimum check node (CN) scheduling in order to improve decoding performance. In particular, we model the CN update process as a multi-armed bandit process with dependent arms and employ a Q-learning scheme for optimizing the CN scheduling policy. In order to reduce the learning complexity, we propose a novel graph-induced CN clustering approach to partition the state space in such a way that dependencies between clusters are minimized. Our results show that compared to other decoding approaches from the literature, the proposed reinforcement learning scheme not only significantly improves the decoding performance, but also reduces the decoding complexity dramatically once the scheduling policy is learned. △ Less

Submitted 17 October, 2020; v1 submitted 12 October, 2020; originally announced October 2020.

Comments: Accepted for publication in 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada

Private Function Computation for Noncolluding Coded Databases

Authors: Sarah A. Obead, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

Abstract: Private computation in a distributed storage system (DSS) is a generalization of the private information retrieval (PIR) problem. In such setting a user wishes to compute a function of $f$ messages stored in $n$ noncolluding coded databases, i.e., databases storing data encoded with an $[n,k]$ linear storage code, while revealing no information about the desired function to the databases. We consi… ▽ More Private computation in a distributed storage system (DSS) is a generalization of the private information retrieval (PIR) problem. In such setting a user wishes to compute a function of $f$ messages stored in $n$ noncolluding coded databases, i.e., databases storing data encoded with an $[n,k]$ linear storage code, while revealing no information about the desired function to the databases. We consider the problem of private polynomial computation (PPC). In PPC, a user wishes to compute a multivariate polynomial of degree at most $g$ over $f$ variables (or messages) stored in multiple databases. First, we consider the private computation of polynomials of degree $g=1$, i.e., private linear computation (PLC) for coded databases. In PLC, a user wishes to compute a linear combination over the $f$ messages while kee** the coefficients of the desired linear combination hidden from the database. For a linearly encoded DSS, we present a capacity-achieving PLC scheme and show that the PLC capacity, which is the ratio of the desired amount of information and the total amount of downloaded information, matches the maximum distance separable coded capacity of PIR for a large class of linear storage codes. Then, we consider private computation of higher degree polynomials, i.e., $g>1$. For this setup, we construct two novel PPC schemes. In the first scheme, we consider Reed-Solomon coded databases with Lagrange encoding, which leverages ideas from recently proposed star-product PIR and Lagrange coded computation. The second scheme considers the special case of coded databases with systematic Lagrange encoding. Both schemes yield improved rates, while asymptotically, as $f\rightarrow \infty$, the systematic scheme gives a significantly better computation retrieval rate compared to all known schemes up to some storage code rate that depends on the maximum degree of the candidate polynomials. △ Less

Submitted 4 August, 2021; v1 submitted 22 March, 2020; originally announced March 2020.

Comments: 41 pages, 4 figures, 11 tables, submitted for publication. Some overlap with arXiv:1810.04230, arXiv:1901.10286

Secure Distributed Storage: Rate-Privacy Trade-Off and XOR-Based Coding Scheme

Authors: Remi A. Chou, Joerg Kliewer

Abstract: We consider the problem of storing data in a distributed manner over $T$ servers. We require the data (i) to be recoverable from the $T$ servers, and (ii) to remain private from any $T-1$ colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at the $T-1$ colluding servers. For this model, we determine (i) the fundamental tr… ▽ More We consider the problem of storing data in a distributed manner over $T$ servers. We require the data (i) to be recoverable from the $T$ servers, and (ii) to remain private from any $T-1$ colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at the $T-1$ colluding servers. For this model, we determine (i) the fundamental trade-off between storage size and the level of desired privacy, (ii) the optimal amount of local randomness necessary at the encoder, and (iii)~an explicit low-complexity coding scheme that solely relies on XOR operations and that asymptotically (with the data size) matches the fundamental limits found. △ Less

Submitted 28 May, 2020; v1 submitted 13 January, 2020; originally announced January 2020.

Comments: 6 pages, full version of paper accepted to the 2020 IEEE International Symposium on Information Theory

Authentication Against a Myopic Adversary

Authors: Allison Beemer, Eric Graves, Joerg Kliewer, Oliver Kosut, Paul Yu

Abstract: We consider keyless authentication for point-to-point communication in the presence of a myopic adversary. In particular, the adversary has access to a non-causal noisy version of the transmission and may use this knowledge to choose the channel state of an arbitrarily-varying channel between legitimate users; the receiver is successful if it either decodes to the correct message or correctly dete… ▽ More We consider keyless authentication for point-to-point communication in the presence of a myopic adversary. In particular, the adversary has access to a non-causal noisy version of the transmission and may use this knowledge to choose the channel state of an arbitrarily-varying channel between legitimate users; the receiver is successful if it either decodes to the correct message or correctly detects adversarial interference. We show that a channel condition called U-overwritability, which allows the adversary to make its false message appear legitimate and untampered with, is a sufficient condition for zero authentication capacity. We present a useful way to compare adversarial channels, and show that once an AVC becomes U-overwritable, it remains U-overwritable for all "less myopic" adversaries. Finally, we show that stochastic encoders are necessary for positive authentication capacity in some cases, and examine in detail a binary adversarial channel that illustrates this necessity. Namely, for this binary channel, we show that when the adversarial channel is degraded with respect to the main channel between users, the no-adversary capacity of the underlying channel is achievable with a deterministic encoder. Otherwise, provided the channel to the adversary is not perfect, a stochastic encoder is necessary for positive authentication capacity; if such an encoder is allowed, the no-adversary capacity is again achievable. △ Less

Submitted 22 January, 2020; v1 submitted 10 January, 2020; originally announced January 2020.

Comments: 22 pages; minor changes to notation and proofs of technical lemmas, results unchanged

Improved Lower Bounds for Pliable Index Coding using Absent Receivers

Authors: Lawrence Ong, Badri N. Vellambi, Jörg Kliewer, Parastoo Sadeghi

Abstract: This paper studies pliable index coding, in which a sender broadcasts information to multiple receivers through a shared broadcast medium, and the receivers each have some message a priori and want any message they do not have. An approach, based on receivers that are absent from the problem, was previously proposed to find lower bounds on the optimal broadcast rate. In this paper, we introduce ne… ▽ More This paper studies pliable index coding, in which a sender broadcasts information to multiple receivers through a shared broadcast medium, and the receivers each have some message a priori and want any message they do not have. An approach, based on receivers that are absent from the problem, was previously proposed to find lower bounds on the optimal broadcast rate. In this paper, we introduce new techniques to obtained better lower bounds, and derive the optimal broadcast rates for new classes of the problems, including all problems with up to four absent receivers. △ Less

Submitted 1 October, 2019; v1 submitted 25 September, 2019; originally announced September 2019.

Comments: An extended version of the same-titled paper submitted to a conference

Optimal-Rate Characterisation for Pliable Index Coding using Absent Receivers

Authors: Lawrence Ong, Badri N. Vellambi, Jörg Kliewer

Abstract: We characterise the optimal broadcast rate for a few classes of pliable-index-coding problems. This is achieved by devising new lower bounds that utilise the set of absent receivers to construct decoding chains with skipped messages. This work complements existing works by considering problems that are not complete-S, i.e., problems considered in this work do not require that all receivers with a… ▽ More We characterise the optimal broadcast rate for a few classes of pliable-index-coding problems. This is achieved by devising new lower bounds that utilise the set of absent receivers to construct decoding chains with skipped messages. This work complements existing works by considering problems that are not complete-S, i.e., problems considered in this work do not require that all receivers with a certain side-information cardinality to be either present or absent from the problem. We show that for a certain class, the set of receivers is critical in the sense that adding any receiver strictly increases the broadcast rate. △ Less

Submitted 25 September, 2019; originally announced September 2019.

Comments: Authors' copy

Journal ref: in Proc. IEEE Int. Symp. Inf. Theory (ISIT), Paris, France, July 7-12 2019, pp. 522-526

Private and Secure Distributed Matrix Multiplication with Flexible Communication Load

Authors: Malihe Aliasgari, Osvaldo Simeone, Joerg Kliewer

Abstract: Large matrix multiplications are central to large-scale machine learning applications. These operations are often carried out on a distributed computing platform with a master server and multiple workers in the cloud operating in parallel. For such distributed platforms, it has been recently shown that coding over the input data matrices can reduce the computational delay, yielding a trade-off bet… ▽ More Large matrix multiplications are central to large-scale machine learning applications. These operations are often carried out on a distributed computing platform with a master server and multiple workers in the cloud operating in parallel. For such distributed platforms, it has been recently shown that coding over the input data matrices can reduce the computational delay, yielding a trade-off between recovery threshold, i.e., the number of workers required to recover the matrix product, and communication load, i.e., the total amount of data to be downloaded from the workers. In this paper, in addition to exact recovery requirements, we impose security and privacy constraints on the data matrices, and study the recovery threshold as a function of the communication load. We first assume that both matrices contain private information and that workers can collude to eavesdrop on the content of these data matrices. For this problem, we introduce a novel class of secure codes, referred to as secure generalized PolyDot (SGPD) codes, that generalize state-of-the-art non-secure codes for matrix multiplication. SGPD codes allow a flexible trade-off between recovery threshold and communication load for a fixed maximum number of colluding workers while providing perfect secrecy for the two data matrices. We then study a connection between secure matrix multiplication and private information retrieval. We specifically assume that one of the data matrices is taken from a public set known to all the workers. In this setup, the identity of the matrix of interest should be kept private from the workers. For this model, we present a variant of generalized PolyDot codes that can guarantee both secrecy of one matrix and privacy for the identity of the other matrix for the case of no colluding servers. △ Less

Submitted 18 December, 2019; v1 submitted 1 September, 2019; originally announced September 2019.

Comments: 12 pages, 9 figures, this submission subsumes arXiv:1901.07705. This work has been submitted to the IEEE for possible publication

LDPC Coded Multiuser Sha** for the Gaussian Multiple Access Channel

Authors: Alexios Balatsoukas-Stimming, Stefano Rini, Joerg Kliewer

Abstract: The joint design of input constellation and low-density parity-check (LDPC) codes to approach the symmetric capacity of the two-user Gaussian multiple access channel is studied. More specifically, multilevel coding is employed at each user to construct a high-order input constellation and the constellations of the users are jointly designed so as to maximize the multiuser sha** gain. At the rece… ▽ More The joint design of input constellation and low-density parity-check (LDPC) codes to approach the symmetric capacity of the two-user Gaussian multiple access channel is studied. More specifically, multilevel coding is employed at each user to construct a high-order input constellation and the constellations of the users are jointly designed so as to maximize the multiuser sha** gain. At the receiver, each layer of the multilevel coding is jointly decoded among users, while successive cancellation is employed across layers. The LDPC code employed by each user in each layer is designed using EXIT charts to support joint decoding among users for the prescribed per-layer rate and SNR. Numerical simulations are provided to validate the proposed constellation and LDPC code designs. △ Less

Submitted 1 May, 2019; originally announced May 2019.

Comments: To be presented at ISIT 2019

Private Polynomial Computation for Noncolluding Coded Databases

Authors: Sarah A. Obead, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

Abstract: We consider private polynomial computation (PPC) over noncolluding coded databases. In such a setting a user wishes to compute a multivariate polynomial of degree at most $g$ over $f$ variables (or messages) stored in multiple databases while revealing no information about the desired polynomial to the databases. We construct two novel PPC schemes, where the first is a generalization of our previo… ▽ More We consider private polynomial computation (PPC) over noncolluding coded databases. In such a setting a user wishes to compute a multivariate polynomial of degree at most $g$ over $f$ variables (or messages) stored in multiple databases while revealing no information about the desired polynomial to the databases. We construct two novel PPC schemes, where the first is a generalization of our previous work in private linear computation for coded databases. In this scheme we consider Reed-Solomon coded databases with Lagrange encoding, which leverages ideas from recently proposed star-product private information retrieval and Lagrange coded computation. The second scheme considers the special case of coded databases with systematic Lagrange encoding. Both schemes yield improved rates compared to the best known schemes from the literature for a small number of messages, while in the asymptotic case the rates match. △ Less

Submitted 7 May, 2019; v1 submitted 29 January, 2019; originally announced January 2019.

Comments: 5 pages, 2 tables, 1 figure, to be presented at 2019 IEEE International Symposium on Information Theory (ISIT)

Journal ref: IEEE International Symposium on Information Theory (ISIT), Paris, France, July 2019, pp. 1677-1681

Distributed and Private Coded Matrix Computation with Flexible Communication Load

Authors: Malihe Aliasgari, Osvaldo Simeone, Joerg Kliewer

Abstract: Tensor operations, such as matrix multiplication, are central to large-scale machine learning applications. For user-driven tasks these operations can be carried out on a distributed computing platform with a master server at the user side and multiple workers in the cloud operating in parallel. For distributed platforms, it has been recently shown that coding over the input data matrices can redu… ▽ More Tensor operations, such as matrix multiplication, are central to large-scale machine learning applications. For user-driven tasks these operations can be carried out on a distributed computing platform with a master server at the user side and multiple workers in the cloud operating in parallel. For distributed platforms, it has been recently shown that coding over the input data matrices can reduce the computational delay, yielding a trade-off between recovery threshold and communication load. In this paper we impose an additional security constraint on the data matrices and assume that workers can collude to eavesdrop on the content of these data matrices. Specifically, we introduce a novel class of secure codes, referred to as secure generalized PolyDot codes, that generalizes previously published non-secure versions of these codes for matrix multiplication. These codes extend the state-of-the-art by allowing a flexible trade-off between recovery threshold and communication load for a fixed maximum number of colluding workers. △ Less

Submitted 22 January, 2019; originally announced January 2019.

Comments: 8 pages, 6 figures, submitted to 2019 IEEE International Symposium on Information Theory (ISIT)

Capacity of Private Linear Computation for Coded Databases

Authors: Sarah A. Obead, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

Abstract: We consider the problem of private linear computation (PLC) in a distributed storage system. In PLC, a user wishes to compute a linear combination of $f$ messages stored in noncolluding databases while revealing no information about the coefficients of the desired linear combination to the databases. In extension of our previous work we employ linear codes to encode the information on the database… ▽ More We consider the problem of private linear computation (PLC) in a distributed storage system. In PLC, a user wishes to compute a linear combination of $f$ messages stored in noncolluding databases while revealing no information about the coefficients of the desired linear combination to the databases. In extension of our previous work we employ linear codes to encode the information on the databases. We show that the PLC capacity, which is the ratio of the desired linear function size and the total amount of downloaded information, matches the maximum distance separable (MDS) coded capacity of private information retrieval for a large class of linear codes that includes MDS codes. In particular, the proposed converse is valid for any number of messages and linear combinations, and the capacity expression depends on the rank of the coefficient matrix obtained from all linear combinations. △ Less

Submitted 9 October, 2018; originally announced October 2018.

Comments: 8 pages. This work has been presented at the 56th Annual Allerton Conference on Communication, Control, and Computing, October 2018

Journal ref: 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA , Oct. 2018, pp. 813-820

Algebraic Optimization of Binary Spatially Coupled Measurement Matrices for Interval Passing

Authors: Salman Habib, Joerg Kliewer

Abstract: We consider binary spatially coupled (SC) low density measurement matrices for low complexity reconstruction of sparse signals via the interval passing algorithm (IPA). The IPA is known to fail due to the presence of harmful sub-structures in the Tanner graph of a binary sparse measurement matrix, so called termatiko sets. In this work we construct array-based (AB) SC sparse measurement matrices v… ▽ More We consider binary spatially coupled (SC) low density measurement matrices for low complexity reconstruction of sparse signals via the interval passing algorithm (IPA). The IPA is known to fail due to the presence of harmful sub-structures in the Tanner graph of a binary sparse measurement matrix, so called termatiko sets. In this work we construct array-based (AB) SC sparse measurement matrices via algebraic lifts of graphs, such that the number of termatiko sets in the Tanner graph is minimized. To this end, we show for the column-weight-three case that the most critical termatiko sets can be removed by eliminating all length-12 cycles associated with the Tanner graph, via algebraic lifting. As a consequence, IPA-based reconstruction with SC measurement matrices is able to provide an almost error free reconstruction for significantly denser signal vectors compared to uncoupled AB LDPC measurement matrices. △ Less

Submitted 15 September, 2018; originally announced September 2018.

Comments: 5 pages, 2 figures, To appear in the Proceedings of 2018 IEEE Information Theory Workshop, Guangzhou, China

Strong Coordination over Noisy Channels

Authors: Sarah A. Obead, Badri N. Vellambi, Jörg Kliewer

Abstract: We study the problem of strong coordination of the actions of two nodes $X$ and $Y$ that communicate over a discrete memoryless channel (DMC) such that the actions follow a prescribed joint probability distribution. We propose two novel random coding schemes and a polar coding scheme for this noisy strong coordination problem, and derive inner bounds for the respective strong coordination capacity… ▽ More We study the problem of strong coordination of the actions of two nodes $X$ and $Y$ that communicate over a discrete memoryless channel (DMC) such that the actions follow a prescribed joint probability distribution. We propose two novel random coding schemes and a polar coding scheme for this noisy strong coordination problem, and derive inner bounds for the respective strong coordination capacity region. The first scheme is a joint coordination-channel coding scheme that utilizes the randomness provided by the DMC to reduce the amount of local randomness required to generate the sequence of actions at Node $Y$. Based on this random coding scheme, we provide a characterization of the capacity region for two special cases of the noisy strong coordination setup, namely, when the actions at Node $Y$ are determined by Node $X$ and when the DMC is a deterministic channel. The second scheme exploits separate coordination and channel coding where local randomness is extracted from the channel after decoding. The third scheme is a joint coordination-channel polar coding scheme for strong coordination. We show that polar codes are able to achieve the established inner bound to the noisy strong coordination capacity region and thus provide a constructive alternative to a random coding proof. Our polar coding scheme also offers a constructive solution to a channel simulation problem where a DMC and shared randomness are employed together to simulate another DMC. Finally, by leveraging the random coding results for this problem, we present an example in which the proposed joint scheme is able to strictly outperform the separate scheme in terms of achievable communication rate for the same amount of injected randomness into both systems. Thus, we establish the sub-optimality of the separation of strong coordination and channel coding with respect to the communication rate over the DMC. △ Less

Submitted 15 August, 2018; originally announced August 2018.

Comments: 30 pages, 9 figures, submitted for publication. arXiv admin note: substantial text overlap with arXiv:1704.08771, arXiv:1710.03210

Journal ref: IEEE Transactions on Information Theory, vol. 67, no. 5, pp. 2716-2738, May 2021

Coded Computation Against Distributed Straggling Channel Decoders in the Cloud for Gaussian Uplink Channels

Authors: **wen Shi, Cong Ling, Osvaldo Simeone, Jörg Kliewer

Abstract: The uplink of a Cloud Radio Access Network (CRAN) architecture is studied, where decoding at the cloud takes place at distributed decoding processors. To mitigate the impact of straggling decoders in the cloud, the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Focusing on Gaussian channels, and assuming the use of lattice codes at the u… ▽ More The uplink of a Cloud Radio Access Network (CRAN) architecture is studied, where decoding at the cloud takes place at distributed decoding processors. To mitigate the impact of straggling decoders in the cloud, the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Focusing on Gaussian channels, and assuming the use of lattice codes at the users, in this paper the maximum user rate is derived such that all the servers can reliably recover the linear combinations of the messages corresponding to the employed linear code at the cloud. Furthermore, two analytical upper bounds on the frame error rate (FER) as a function of the decoding latency are developed, in order to quantify the performance of the cloud's linear code in terms of the tradeoff between FER and decoding latency at the cloud. △ Less

Submitted 12 July, 2018; v1 submitted 29 May, 2018; originally announced May 2018.

A Code Equivalence between Secure Network and Index Coding

Authors: Lawrence Ong, Jörg Kliewer, Badri N. Vellambi, Phee Lep Yeoh

Abstract: A code equivalence between index coding and network coding was established, which shows that any index-coding instance can be mapped to a network-coding instance, for which any index code can be translated to a network code with the same decoding-error performance, and vice versa. Also, any network-coding instance can be mapped to an index-coding instance with a similar code translation. In this p… ▽ More A code equivalence between index coding and network coding was established, which shows that any index-coding instance can be mapped to a network-coding instance, for which any index code can be translated to a network code with the same decoding-error performance, and vice versa. Also, any network-coding instance can be mapped to an index-coding instance with a similar code translation. In this paper, we extend the equivalence to secure index coding and secure network coding, where eavesdroppers are present in the networks, and any code construction needs to guarantee security constraints in addition to decoding-error performance. △ Less

Submitted 26 April, 2018; originally announced April 2018.

Comments: The journal version of conference papers "An Equivalence Between Secure Network and Index Coding" (Globecom-NetCod 2016) and "Secure Network-Index Code Equivalence: Extension to Non-zero Error and Leakage" (ISIT 2018)

MSC Class: 94A15

Achievable Rate of Private Function Retrieval from MDS Coded Databases

Authors: Sarah A. Obead, Jörg Kliewer

Abstract: We study the problem of private function retrieval (PFR) in a distributed storage system. In PFR the user wishes to retrieve a linear combination of $M$ messages stored in non-colluding $(N,K)$ MDS coded databases while revealing no information about the coefficients of the intended linear combination to any of the individual databases. We present an achievable scheme for MDS coded PFR with a rate… ▽ More We study the problem of private function retrieval (PFR) in a distributed storage system. In PFR the user wishes to retrieve a linear combination of $M$ messages stored in non-colluding $(N,K)$ MDS coded databases while revealing no information about the coefficients of the intended linear combination to any of the individual databases. We present an achievable scheme for MDS coded PFR with a rate that matches the capacity for coded private information retrieval derived recently, $R=(1+R_c+R_c^2+\dots+R_c^{M-1})^{-1}=\frac{1-R_c}{1-R_c^M}$, where $R_c=\frac{K}{N}$ is the rate of the MDS code. This achievable rate is tight in some special cases. △ Less

Submitted 22 February, 2018; originally announced February 2018.

Comments: 5 pages, 1 table, submitted for publication

Journal ref: IEEE International Symposium on Information Theory (ISIT), Vail, CO, USA, June 2018, pp. 2117-2121

Finite Blocklength and Dispersion Bounds for the Arbitrarily-Varying Channel

Authors: Oliver Kosut, Joerg Kliewer

Abstract: Finite blocklength and second-order (dispersion) results are presented for the arbitrarily-varying channel (AVC), a classical model wherein an adversary can transmit arbitrary signals into the channel. A novel finite blocklength achievability bound is presented, roughly analogous to the random coding union bound for non-adversarial channels. This finite blocklength bound, along with a known conver… ▽ More Finite blocklength and second-order (dispersion) results are presented for the arbitrarily-varying channel (AVC), a classical model wherein an adversary can transmit arbitrary signals into the channel. A novel finite blocklength achievability bound is presented, roughly analogous to the random coding union bound for non-adversarial channels. This finite blocklength bound, along with a known converse bound, is used to derive bounds on the dispersion of discrete memoryless AVCs without shared randomness, and with cost constraints on the input and the state. These bounds are tight for many channels of interest, including the binary symmetric AVC. However, the bounds are not tight if the deterministic and random code capacities differ. △ Less

Submitted 10 January, 2018; originally announced January 2018.

Comments: 7 pages, full version of paper submitted to the 2018 IEEE International Symposium on Information Theory

A Generalized Algebraic Approach to Optimizing SC-LDPC Codes

Authors: Allison Beemer, Salman Habib, Christine A. Kelley, Joerg Kliewer

Abstract: Spatially coupled low-density parity-check (SC-LDPC) codes are sparse graph codes that have recently become of interest due to their capacity-approaching performance on memoryless binary input channels. In this paper, we unify all existing SC-LDPC code construction methods under a new generalized description of SC-LDPC codes based on algebraic lifts of graphs. We present an improved low-complexity… ▽ More Spatially coupled low-density parity-check (SC-LDPC) codes are sparse graph codes that have recently become of interest due to their capacity-approaching performance on memoryless binary input channels. In this paper, we unify all existing SC-LDPC code construction methods under a new generalized description of SC-LDPC codes based on algebraic lifts of graphs. We present an improved low-complexity counting method for the special case of $(3,3)$-absorbing sets for array-based SC-LDPC codes, which we then use to optimize permutation assignments in SC-LDPC code construction. We show that codes constructed in this way are able to outperform previously published constructions, in terms of the number of dominant absorbing sets and with respect to both standard and windowed decoding. △ Less

Submitted 10 October, 2017; originally announced October 2017.

Comments: 8 pages, to appear in the Proceedings of the 55th Annual Allerton Conference on Communication, Control, and Computing, 2017

Joint Coordination-Channel Coding for Strong Coordination over Noisy Channels Based on Polar Codes

Authors: Sarah A. Obead, Jörg Kliewer, Badri N. Vellambi

Abstract: We construct a joint coordination-channel polar coding scheme for strong coordination of actions between two agents $\mathsf X$ and $\mathsf Y$, which communicate over a discrete memoryless channel (DMC) such that the joint distribution of actions follows a prescribed probability distribution. We show that polar codes are able to achieve our previously established inner bound to the strong noisy c… ▽ More We construct a joint coordination-channel polar coding scheme for strong coordination of actions between two agents $\mathsf X$ and $\mathsf Y$, which communicate over a discrete memoryless channel (DMC) such that the joint distribution of actions follows a prescribed probability distribution. We show that polar codes are able to achieve our previously established inner bound to the strong noisy coordination capacity region and thus provide a constructive alternative to a random coding proof. Our polar coding scheme also offers a constructive solution to a channel simulation problem where a DMC and shared randomness are together employed to simulate another DMC. In particular, our proposed solution is able to utilize the randomness of the DMC to reduce the amount of local randomness required to generate the sequence of actions at agent $\mathsf Y$. By leveraging our earlier random coding results for this problem, we conclude that the proposed joint coordination-channel coding scheme strictly outperforms a separate scheme in terms of achievable communication rate for the same amount of injected randomness into both systems. △ Less

Submitted 9 October, 2017; originally announced October 2017.

Comments: 8 pages, 6 figures. Accepted in the 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2017

Journal ref: 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA, Oct 2017, pp. 580-587

Coded Computation Against Processing Delays for Virtualized Cloud-Based Channel Decoding

Authors: Malihe Aliasgari, Jörg Kliewer, Osvaldo Simeone

Abstract: The uplink of a cloud radio access network architecture is studied in which decoding at the cloud takes place via network function virtualization on commercial off-the-shelf servers. In order to mitigate the impact of straggling decoders in this platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing them to the decoding… ▽ More The uplink of a cloud radio access network architecture is studied in which decoding at the cloud takes place via network function virtualization on commercial off-the-shelf servers. In order to mitigate the impact of straggling decoders in this platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Transmission of a single frame is considered first, and upper bounds on the resulting frame unavailability probability as a function of the decoding latency are derived by assuming a binary symmetric channel for uplink communications. Then, the analysis is extended to account for random frame arrival times. In this case, the trade-off between average decoding latency and the frame error rate is studied for two different queuing policies, whereby the servers carry out per-frame decoding or continuous decoding, respectively. Numerical examples demonstrate that the bounds are useful tools for code design and that coding is instrumental in obtaining a desirable compromise between decoding latency and reliability. △ Less

Submitted 28 August, 2018; v1 submitted 4 September, 2017; originally announced September 2017.

Comments: 11 pages and 12 figures, Submitted

MSC Class: 68P30

Strong Converses Are Just Edge Removal Properties

Authors: Oliver Kosut, Joerg Kliewer

Abstract: This paper explores the relationship between two ideas in network information theory: edge removal and strong converses. Edge removal properties state that if an edge of small capacity is removed from a network, the capacity region does not change too much. Strong converses state that, for rates outside the capacity region, the probability of error converges to 1 as the blocklength goes to infinit… ▽ More This paper explores the relationship between two ideas in network information theory: edge removal and strong converses. Edge removal properties state that if an edge of small capacity is removed from a network, the capacity region does not change too much. Strong converses state that, for rates outside the capacity region, the probability of error converges to 1 as the blocklength goes to infinity. Various notions of edge removal and strong converse are defined, depending on how edge capacity and error probability scale with blocklength, and relations between them are proved. Each class of strong converse implies a specific class of edge removal. The opposite directions are proved for deterministic networks. Furthermore, a technique based on a novel, causal version of the blowing-up lemma is used to prove that for discrete memoryless networks, the weak edge removal property--that the capacity region changes continuously as the capacity of an edge vanishes--is equivalent to the exponentially strong converse--that outside the capacity region, the probability of error goes to 1 exponentially fast. This result is used to prove exponentially strong converses for several examples, including the discrete 2-user interference channel with strong interference, with only a small variation from traditional weak converse proofs. △ Less

Submitted 11 December, 2018; v1 submitted 25 June, 2017; originally announced June 2017.

Comments: (v4) Addition of Table I clarifying notation, corrected proof of Proposition 3, and other minor improvements

Strong Coordination over Noisy Channels: Is Separation Sufficient?

Authors: Sarah A. Obead, Badri N. Vellambi, Jörg Kliewer

Abstract: We study the problem of strong coordination of actions of two agents $X$ and $Y$ that communicate over a noisy communication channel such that the actions follow a given joint probability distribution. We propose two novel schemes for this noisy strong coordination problem, and derive inner bounds for the underlying strong coordination capacity region. The first scheme is a joint coordination-chan… ▽ More We study the problem of strong coordination of actions of two agents $X$ and $Y$ that communicate over a noisy communication channel such that the actions follow a given joint probability distribution. We propose two novel schemes for this noisy strong coordination problem, and derive inner bounds for the underlying strong coordination capacity region. The first scheme is a joint coordination-channel coding scheme that utilizes the randomness provided by the communication channel to reduce the local randomness required in generating the action sequence at agent $Y$. The second scheme exploits separate coordination and channel coding where local randomness is extracted from the channel after decoding. Finally, we present an example in which the joint scheme is able to outperform the separate scheme in terms of coordination rate. △ Less

Submitted 27 April, 2017; originally announced April 2017.

Comments: 9 pages, 4 figures. An extended version of a paper accepted for the IEEE International Symposium on Information Theory (ISIT), 2017

Journal ref: IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, June 2017, pp. 2840-2844

An Equivalence Between Secure Network and Index Coding

Authors: Lawrence Ong, Badri N. Vellambi, Jörg Kliewer, Phee Lep Yeoh

Abstract: We extend the equivalence between network coding and index coding by Effros, El Rouayheb, and Langberg to the secure communication setting in the presence of an eavesdropper. Specifically, we show that the most general versions of secure network-coding setup by Chan and Grant and the secure index-coding setup by Dau, Skachek, and Chee, which also include the randomised encoding setting, are equiva… ▽ More We extend the equivalence between network coding and index coding by Effros, El Rouayheb, and Langberg to the secure communication setting in the presence of an eavesdropper. Specifically, we show that the most general versions of secure network-coding setup by Chan and Grant and the secure index-coding setup by Dau, Skachek, and Chee, which also include the randomised encoding setting, are equivalent. △ Less

Submitted 15 February, 2017; originally announced February 2017.

Journal ref: Proceedings of the 2016 IEEE Globecom Workshop on Network Coding and Applications (NetCod), Washington, USA, Dec. 4-8, 2016

Empirical and Strong Coordination via Soft Covering with Polar Codes

Authors: Remi A. Chou, Matthieu Bloch, Joerg Kliewer

Abstract: We design polar codes for empirical coordination and strong coordination in two-node networks. Our constructions hinge on the fact that polar codes enable explicit low-complexity schemes for soft covering. We leverage this property to propose explicit and low-complexity coding schemes that achieve the capacity regions of both empirical coordination and strong coordination for sequences of actions… ▽ More We design polar codes for empirical coordination and strong coordination in two-node networks. Our constructions hinge on the fact that polar codes enable explicit low-complexity schemes for soft covering. We leverage this property to propose explicit and low-complexity coding schemes that achieve the capacity regions of both empirical coordination and strong coordination for sequences of actions taking value in an alphabet of prime cardinality. Our results improve previously known polar coding schemes, which (i) were restricted to uniform distributions and to actions obtained via binary symmetric channels for strong coordination, (ii) required a non-negligible amount of common randomness for empirical coordination, and (iii) assumed that the simulation of discrete memoryless channels could be perfectly implemented. As a by-product of our results, we obtain a polar coding scheme that achieves channel resolvability for an arbitrary discrete memoryless channel whose input alphabet has prime cardinality. △ Less

Submitted 6 June, 2018; v1 submitted 30 August, 2016; originally announced August 2016.

Comments: 14 pages, two-column, 5 figures, accepted to IEEE Transactions on Information Theory

Coded Network Function Virtualization: Fault Tolerance via In-Network Coding

Authors: Ali Al-Shuwaili, Osvaldo Simeone, Joerg Kliewer, Petar Popovski

Abstract: Network Function Virtualization (NFV) prescribes the instantiation of network functions on general-purpose network devices, such as servers and switches. While yielding a more flexible and cost-effective network architecture, NFV is potentially limited by the fact that commercial off-the-shelf hardware is less reliable than the dedicated network elements used in conventional cellular deployments.… ▽ More Network Function Virtualization (NFV) prescribes the instantiation of network functions on general-purpose network devices, such as servers and switches. While yielding a more flexible and cost-effective network architecture, NFV is potentially limited by the fact that commercial off-the-shelf hardware is less reliable than the dedicated network elements used in conventional cellular deployments. The typical solution for this problem is to duplicate network functions across geographically distributed hardware in order to ensure diversity. In contrast, this letter proposes to leverage channel coding in order to enhance the robustness on NFV to hardware failure. The proposed approach targets the network function of uplink channel decoding, and builds on the algebraic structure of the encoded data frames in order to perform in-network coding on the signals to be processed at different servers. The key principles underlying the proposed coded NFV approach are presented for a simple embodiment and extensions are discussed. Numerical results demonstrate the potential gains obtained with the proposed scheme as compared to the conventional diversity-based fault-tolerant scheme in terms of error probability. △ Less

Submitted 16 September, 2016; v1 submitted 12 August, 2016; originally announced August 2016.

Comments: 4 pages, 4 figures. To appear in IEEE Wireless Communication Letter

Strong Coordination over Multi-hop Line Networks

Authors: Badri N Vellambi, Joerg Kliewer, Matthieu Bloch

Abstract: We analyze the problem of strong coordination over a multi-hop line network in which the node initiating the coordination is a terminal network node. We assume that each node has access to a certain amount of randomness that is local to the node, and that the nodes share some common randomness, which are used together with explicit hop-by-hop communication to achieve strong coordination. We derive… ▽ More We analyze the problem of strong coordination over a multi-hop line network in which the node initiating the coordination is a terminal network node. We assume that each node has access to a certain amount of randomness that is local to the node, and that the nodes share some common randomness, which are used together with explicit hop-by-hop communication to achieve strong coordination. We derive the trade-offs among the required rates of communication on the network links, the rates of local randomness available to network nodes, and the rate of common randomness to realize strong coordination. We present an achievable coding scheme built using multiple layers of channel resolvability codes, and establish several settings in which this scheme is proven to offer the best possible trade-offs. △ Less

Submitted 8 April, 2016; v1 submitted 29 February, 2016; originally announced February 2016.

Comments: 35 pages, 9 Figures, 4 Tables. A part of this work were published in the 2015 IEEE Information Theory Workshop, and a part was accepted for publication in the 50th Annual Conference on Information Sciences and Systems

Lossy Compression with Near-uniform Encoder Outputs

Authors: Badri N Vellambi, Joerg Kliewer, Matthieu Bloch

Abstract: It is well known that lossless compression of a discrete memoryless source with near-uniform encoder output is possible at a rate above its entropy if and only if the encoder is randomized. This work focuses on deriving conditions for near-uniform encoder output(s) in the Wyner-Ziv and the distributed lossy compression problems. We show that in the Wyner-Ziv problem, near-uniform encoder output an… ▽ More It is well known that lossless compression of a discrete memoryless source with near-uniform encoder output is possible at a rate above its entropy if and only if the encoder is randomized. This work focuses on deriving conditions for near-uniform encoder output(s) in the Wyner-Ziv and the distributed lossy compression problems. We show that in the Wyner-Ziv problem, near-uniform encoder output and operation close to the WZ-rate limit is simultaneously possible, whereas in the distributed lossy compression problem, jointly near-uniform outputs is achievable in the interior of the distributed lossy compression rate region if the sources share non-trivial Gács-Körner common information. △ Less

Submitted 9 July, 2016; v1 submitted 22 February, 2016; originally announced February 2016.

Comments: Submitted to the 2016 IEEE International Symposium on Information Theory (11 Pages, 3 Figures)

Secure Index Coding: Existence and Construction

Authors: Lawrence Ong, Badri N. Vellambi, Phee Lep Yeoh, Jörg Kliewer, **hong Yuan

Abstract: We investigate the construction of weakly-secure index codes for a sender to send messages to multiple receivers with side information in the presence of an eavesdropper. We derive a sufficient and necessary condition for the existence of index codes that are secure against an eavesdropper with access to any subset of messages of cardinality $t$, for any fixed $t$. In contrast to the benefits of u… ▽ More We investigate the construction of weakly-secure index codes for a sender to send messages to multiple receivers with side information in the presence of an eavesdropper. We derive a sufficient and necessary condition for the existence of index codes that are secure against an eavesdropper with access to any subset of messages of cardinality $t$, for any fixed $t$. In contrast to the benefits of using random keys in secure network coding, we prove that random keys do not promote security in three classes of index-coding instances. △ Less

Submitted 31 May, 2016; v1 submitted 2 February, 2016; originally announced February 2016.

Comments: Author final manuscript (to be presented at the 2016 IEEE International Symposium on Information Theory)

An Information Theoretic Approach Towards Assessing Perceptual Audio Quality using EEG

Authors: Ketan Mehta, Jörg Kliewer

Abstract: In this paper, we propose a novel information theoretic model to interpret the entire "transmission chain" comprising stimulus generation, brain processing by the human subject, and the electroencephalograph (EEG) response measurements as a nonlinear, time-varying communication channel with memory. We use mutual information (MI) as a measure to assess audio quality perception by directly measuring… ▽ More In this paper, we propose a novel information theoretic model to interpret the entire "transmission chain" comprising stimulus generation, brain processing by the human subject, and the electroencephalograph (EEG) response measurements as a nonlinear, time-varying communication channel with memory. We use mutual information (MI) as a measure to assess audio quality perception by directly measuring the brainwave responses of the human subjects using a high resolution EEG. Our focus here is on audio where the quality is impaired by time varying distortions. In particular, we conduct experiments where subjects are presented with audio whose quality varies with time between different possible quality levels. The recorded EEG measurements can be modeled as a multidimensional Gaussian mixture model (GMM). In order to make the computation of the MI feasible, we present a novel low-complexity approximation technique for the differential entropy of the multidimensional GMM. We find the proposed information theoretic approach to be successful in quantifying subjective audio quality perception, with the results being consistent across different music sequences and distortion types. △ Less

Submitted 13 September, 2015; originally announced September 2015.

Comments: 29 pages, 10 figures, IEEE Transactions on Molecular, Biological and Multi-scale Communications

Coding Schemes for Achieving Strong Secrecy at Negligible Cost

Authors: Remi A. Chou, Badri Vellambi, Matthieu Bloch, Joerg Kliewer

Abstract: We study the problem of achieving strong secrecy over wiretap channels at negligible cost, in the sense of maintaining the overall communication rate of the same channel without secrecy constraints. Specifically, we propose and analyze two source-channel coding architectures, in which secrecy is achieved by multiplexing public and confidential messages. In both cases, our main contribution is to s… ▽ More We study the problem of achieving strong secrecy over wiretap channels at negligible cost, in the sense of maintaining the overall communication rate of the same channel without secrecy constraints. Specifically, we propose and analyze two source-channel coding architectures, in which secrecy is achieved by multiplexing public and confidential messages. In both cases, our main contribution is to show that secrecy can be achieved without compromising communication rate and by requiring only randomness of asymptotically vanishing rate. Our first source-channel coding architecture relies on a modified wiretap channel code, in which randomization is performed using the output of a source code. In contrast, our second architecture relies on a standard wiretap code combined with a modified source code termed uniform compression code, in which a small shared secret seed is used to enhance the uniformity of the source code output. We carry out a detailed analysis of uniform compression codes and characterize the optimal size of the shared seed. △ Less

Submitted 5 December, 2016; v1 submitted 31 August, 2015; originally announced August 2015.

Comments: 15 pages, two-column, 5 figures, accepted to IEEE Transactions on Information Theory

Communication Efficient Secret Sharing

Authors: Wentao Huang, Michael Langberg, Joerg Kliewer, Jehoshua Bruck

Abstract: A secret sharing scheme is a method to store information securely and reliably. Particularly, in a threshold secret sharing scheme, a secret is encoded into $n$ shares, such that any set of at least $t_1$ shares suffice to decode the secret, and any set of at most $t_2 < t_1$ shares reveal no information about the secret. Assuming that each party holds a share and a user wishes to decode the secre… ▽ More A secret sharing scheme is a method to store information securely and reliably. Particularly, in a threshold secret sharing scheme, a secret is encoded into $n$ shares, such that any set of at least $t_1$ shares suffice to decode the secret, and any set of at most $t_2 < t_1$ shares reveal no information about the secret. Assuming that each party holds a share and a user wishes to decode the secret by receiving information from a set of parties; the question we study is how to minimize the amount of communication between the user and the parties. We show that the necessary amount of communication, termed "decoding bandwidth", decreases as the number of parties that participate in decoding increases. We prove a tight lower bound on the decoding bandwidth, and construct secret sharing schemes achieving the bound. Particularly, we design a scheme that achieves the optimal decoding bandwidth when $d$ parties participate in decoding, universally for all $t_1 \le d \le n$. The scheme is based on Shamir's secret sharing scheme and preserves its simplicity and efficiency. In addition, we consider secure distributed storage where the proposed communication efficient secret sharing schemes further improve disk access complexity during decoding. △ Less

Submitted 1 April, 2016; v1 submitted 27 May, 2015; originally announced May 2015.

Comments: submitted to the IEEE Transactions on Information Theory. New references and a new construction added

Lossy Compression with Privacy Constraints: Optimality of Polar Codes

Authors: Farshid Mokhtarinezhad, Joerg Kliewer, Osvaldo Simeone

Abstract: A lossy source coding problem with privacy constraint is studied in which two correlated discrete sources $X$ and $Y$ are compressed into a reconstruction $\hat{X}$ with some prescribed distortion $D$. In addition, a privacy constraint is specified as the equivocation between the lossy reconstruction $\hat{X}$ and $Y$. This models the situation where a certain amount of source information from one… ▽ More A lossy source coding problem with privacy constraint is studied in which two correlated discrete sources $X$ and $Y$ are compressed into a reconstruction $\hat{X}$ with some prescribed distortion $D$. In addition, a privacy constraint is specified as the equivocation between the lossy reconstruction $\hat{X}$ and $Y$. This models the situation where a certain amount of source information from one user is provided as utility (given by the fidelity of its reconstruction) to another user or the public, while some other correlated part of the source information $Y$ must be kept private. In this work, we show that polar codes are able, possibly with the aid of time sharing, to achieve any point in the optimal rate-distortion-equivocation region identified by Yamamoto, thus providing a constructive scheme that obtains the optimal tradeoff between utility and privacy in this framework. △ Less

Submitted 21 April, 2015; originally announced April 2015.

Comments: Submitted for publication