Showing 51–100 of 120 results for author: Ramchandran, K

Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates

Authors: Dong Yin, Yudong Chen, Kannan Ramchandran, Peter Bartlett

Abstract: In large-scale distributed learning, security issues have become increasingly important. Particularly in a decentralized environment, some computing units may behave abnormally, or even exhibit Byzantine failures -- arbitrary and potentially adversarial behavior. In this paper, we develop distributed learning algorithms that are provably robust against such failures, with a focus on achieving opti… ▽ More In large-scale distributed learning, security issues have become increasingly important. Particularly in a decentralized environment, some computing units may behave abnormally, or even exhibit Byzantine failures -- arbitrary and potentially adversarial behavior. In this paper, we develop distributed learning algorithms that are provably robust against such failures, with a focus on achieving optimal statistical performance. A main result of this work is a sharp analysis of two robust distributed gradient descent algorithms based on median and trimmed mean operations, respectively. We prove statistical error rates for three kinds of population loss functions: strongly convex, non-strongly convex, and smooth non-convex. In particular, these algorithms are shown to achieve order-optimal statistical error rates for strongly convex losses. To achieve better communication efficiency, we further propose a median-based distributed algorithm that is provably robust, and uses only one communication round. For strongly convex quadratic loss, we show that this algorithm achieves the same optimal error rate as the robust distributed gradient descent algorithms. △ Less

Submitted 25 February, 2021; v1 submitted 5 March, 2018; originally announced March 2018.

Comments: ICML 2018

Approximate Ranking from Pairwise Comparisons

Authors: Reinhard Heckel, Max Simchowitz, Kannan Ramchandran, Martin J. Wainwright

Abstract: A common problem in machine learning is to rank a set of n items based on pairwise comparisons. Here ranking refers to partitioning the items into sets of pre-specified sizes according to their scores, which includes identification of the top-k items as the most prominent special case. The score of a given item is defined as the probability that it beats a randomly chosen other item. Finding an ex… ▽ More A common problem in machine learning is to rank a set of n items based on pairwise comparisons. Here ranking refers to partitioning the items into sets of pre-specified sizes according to their scores, which includes identification of the top-k items as the most prominent special case. The score of a given item is defined as the probability that it beats a randomly chosen other item. Finding an exact ranking typically requires a prohibitively large number of comparisons, but in practice, approximate rankings are often adequate. Accordingly, we study the problem of finding approximate rankings from pairwise comparisons. We analyze an active ranking algorithm that counts the number of comparisons won, and decides whether to stop or which pair of items to compare next, based on confidence intervals computed from the data collected in previous steps. We show that this algorithm succeeds in recovering approximate rankings using a number of comparisons that is close to optimal up to logarithmic factors. We also present numerical results, showing that in practice, approximation can drastically reduce the number of comparisons required to estimate a ranking. △ Less

Submitted 4 January, 2018; originally announced January 2018.

Comments: AISTATS 2017

A Sequential Approximation Framework for Coded Distributed Optimization

Authors: **gge Zhu, Ye Pu, Vipul Gupta, Claire Tomlin, Kannan Ramchandran

Abstract: Building on the previous work of Lee et al. and Ferdinand et al. on coded computation, we propose a sequential approximation framework for solving optimization problems in a distributed manner. In a distributed computation system, latency caused by individual processors ("stragglers") usually causes a significant delay in the overall process. The proposed method is powered by a sequential computat… ▽ More Building on the previous work of Lee et al. and Ferdinand et al. on coded computation, we propose a sequential approximation framework for solving optimization problems in a distributed manner. In a distributed computation system, latency caused by individual processors ("stragglers") usually causes a significant delay in the overall process. The proposed method is powered by a sequential computation scheme, which is designed specifically for systems with stragglers. This scheme has the desirable property that the user is guaranteed to receive useful (approximate) computation results whenever a processor finishes its subtask, even in the presence of uncertain latency. In this paper, we give a coding theorem for sequentially computing matrix-vector multiplications, and the optimality of this coding scheme is also established. As an application of the results, we demonstrate solving optimization problems using a sequential approximation approach, which accelerates the algorithm in a distributed system with stragglers. △ Less

Submitted 24 October, 2017; originally announced October 2017.

Comments: presented in 55th Annual Allerton Conference on Communication, Control, and Computing, Oct. 2017

Gradient Diversity: a Key Ingredient for Scalable Distributed Learning

Authors: Dong Yin, Ashwin Pananjady, Max Lam, Dimitris Papailiopoulos, Kannan Ramchandran, Peter Bartlett

Abstract: It has been experimentally observed that distributed implementations of mini-batch stochastic gradient descent (SGD) algorithms exhibit speedup saturation and decaying generalization ability beyond a particular batch-size. In this work, we present an analysis hinting that high similarity between concurrently processed gradients may be a cause of this performance degradation. We introduce the notio… ▽ More It has been experimentally observed that distributed implementations of mini-batch stochastic gradient descent (SGD) algorithms exhibit speedup saturation and decaying generalization ability beyond a particular batch-size. In this work, we present an analysis hinting that high similarity between concurrently processed gradients may be a cause of this performance degradation. We introduce the notion of gradient diversity that measures the dissimilarity between concurrent gradient updates, and show its key role in the performance of mini-batch SGD. We prove that on problems with high gradient diversity, mini-batch SGD is amenable to better speedups, while maintaining the generalization performance of serial (one sample) SGD. We further establish lower bounds on convergence where mini-batch SGD slows down beyond a particular batch-size, solely due to the lack of gradient diversity. We provide experimental evidence indicating the key role of gradient diversity in distributed learning, and discuss how heuristics like dropout, Langevin dynamics, and quantization can improve it. △ Less

Submitted 6 January, 2018; v1 submitted 18 June, 2017; originally announced June 2017.

The Sample Complexity of Online One-Class Collaborative Filtering

Authors: Reinhard Heckel, Kannan Ramchandran

Abstract: We consider the online one-class collaborative filtering (CF) problem that consists of recommending items to users over time in an online fashion based on positive ratings only. This problem arises when users respond only occasionally to a recommendation with a positive rating, and never with a negative one. We study the impact of the probability of a user responding to a recommendation, p_f, on t… ▽ More We consider the online one-class collaborative filtering (CF) problem that consists of recommending items to users over time in an online fashion based on positive ratings only. This problem arises when users respond only occasionally to a recommendation with a positive rating, and never with a negative one. We study the impact of the probability of a user responding to a recommendation, p_f, on the sample complexity, i.e., the number of ratings required to make `good' recommendations, and ask whether receiving positive and negative ratings, instead of positive ratings only, improves the sample complexity. Both questions arise in the design of recommender systems. We introduce a simple probabilistic user model, and analyze the performance of an online user-based CF algorithm. We prove that after an initial cold start phase, where recommendations are invested in exploring the user's preferences, this algorithm makes---up to a fraction of the recommendations required for updating the user's preferences---perfect recommendations. The number of ratings required for the cold start phase is nearly proportional to 1/p_f, and that for updating the user's preferences is essentially independent of p_f. As a consequence we find that, receiving positive and negative ratings instead of only positive ones improves the number of ratings required for initial exploration by a factor of 1/p_f, which can be significant. △ Less

Submitted 31 May, 2017; originally announced June 2017.

Comments: ICML 2017

Fundamental Limits of DNA Storage Systems

Authors: Reinhard Heckel, Ilan Shomorony, Kannan Ramchandran, David N. C. Tse

Abstract: Due to its longevity and enormous information density, DNA is an attractive medium for archival storage. In this work, we study the fundamental limits and tradeoffs of DNA-based storage systems under a simple model, motivated by current technological constraints on DNA synthesis and sequencing. Our model captures two key distinctive aspects of DNA storage systems: (1) the data is written onto many… ▽ More Due to its longevity and enormous information density, DNA is an attractive medium for archival storage. In this work, we study the fundamental limits and tradeoffs of DNA-based storage systems under a simple model, motivated by current technological constraints on DNA synthesis and sequencing. Our model captures two key distinctive aspects of DNA storage systems: (1) the data is written onto many short DNA molecules that are stored in an unordered way and (2) the data is read by randomly sampling from this DNA pool. Under this model, we characterize the storage capacity, and show that a simple index-based coding scheme is optimal. △ Less

Submitted 12 May, 2017; originally announced May 2017.

Comments: To appear in Proc. of IEEE International Symposium on Information Theory (ISIT). Slightly extended version containing the proofs

Density Evolution on a Class of Smeared Random Graphs: A Theoretical Framework for Fast MRI

Authors: Kabir Chandrasekher, Orhan Ocal, Kannan Ramchandran

Abstract: We introduce a new ensemble of random bipartite graphs, which we term the `smearing ensemble', where each left node is connected to some number of consecutive right nodes. Such graphs arise naturally in the recovery of sparse wavelet coefficients when signal acquisition is in the Fourier domain, such as in magnetic resonance imaging (MRI). Graphs from this ensemble exhibit small, structured cycles… ▽ More We introduce a new ensemble of random bipartite graphs, which we term the `smearing ensemble', where each left node is connected to some number of consecutive right nodes. Such graphs arise naturally in the recovery of sparse wavelet coefficients when signal acquisition is in the Fourier domain, such as in magnetic resonance imaging (MRI). Graphs from this ensemble exhibit small, structured cycles with high probability, rendering current techniques for determining iterative decoding thresholds inapplicable. In this paper, we develop a theoretical platform to analyze and evaluate the effects of smearing-based structure. Despite the existence of these small cycles, we derive exact density evolution recurrences for iterative decoding on graphs with smear-length two. Further, we give lower bounds on the performance of a much larger class from the smearing ensemble, and provide numerical experiments showing tight agreement between empirical thresholds and those determined by our bounds. Finally, we describe a system architecture to recover sparse wavelet representations in the MRI setting, giving explicit thresholds on the minimum number of Fourier samples needing to be acquired for the $1$-stage Haar wavelet setting. In particular, we show that $K$-sparse $1$-stage Haar wavelet coefficients of an $n$-dimensional signal can be recovered using $2.63K$ Fourier domain samples asymptotically using $\mathcal{O}(K\log{K})$ operations. △ Less

Submitted 6 May, 2017; originally announced May 2017.

Learning Mixtures of Sparse Linear Regressions Using Sparse Graph Codes

Authors: Dong Yin, Ramtin Pedarsani, Yudong Chen, Kannan Ramchandran

Abstract: In this paper, we consider the mixture of sparse linear regressions model. Let $β^{(1)},\ldots,β^{(L)}\in\mathbb{C}^n$ be $ L $ unknown sparse parameter vectors with a total of $ K $ non-zero coefficients. Noisy linear measurements are obtained in the form $y_i={x}_i^H β^{(\ell_i)} + w_i$, each of which is generated randomly from one of the sparse vectors with the label $ \ell_i $ unknown. The goa… ▽ More In this paper, we consider the mixture of sparse linear regressions model. Let $β^{(1)},\ldots,β^{(L)}\in\mathbb{C}^n$ be $ L $ unknown sparse parameter vectors with a total of $ K $ non-zero coefficients. Noisy linear measurements are obtained in the form $y_i={x}_i^H β^{(\ell_i)} + w_i$, each of which is generated randomly from one of the sparse vectors with the label $ \ell_i $ unknown. The goal is to estimate the parameter vectors efficiently with low sample and computational costs. This problem presents significant challenges as one needs to simultaneously solve the demixing problem of recovering the labels $ \ell_i $ as well as the estimation problem of recovering the sparse vectors $ β^{(\ell)} $. Our solution to the problem leverages the connection between modern coding theory and statistical inference. We introduce a new algorithm, Mixed-Coloring, which samples the mixture strategically using query vectors $ {x}_i $ constructed based on ideas from sparse graph codes. Our novel code design allows for both efficient demixing and parameter estimation. In the noiseless setting, for a constant number of sparse parameter vectors, our algorithm achieves the order-optimal sample and time complexities of $Θ(K)$. In the presence of Gaussian noise, for the problem with two parameter vectors (i.e., $L=2$), we show that the Robust Mixed-Coloring algorithm achieves near-optimal $Θ(K polylog(n))$ sample and time complexities. When $K=O(n^α)$ for some constant $α\in(0,1)$ (i.e., $K$ is sublinear in $n$), we can achieve sample and time complexities both sublinear in the ambient dimension. In one of our experiments, to recover a mixture of two regressions with dimension $n=500$ and sparsity $K=50$, our algorithm is more than $300$ times faster than EM algorithm, with about one third of its sample cost. △ Less

Submitted 2 August, 2018; v1 submitted 2 March, 2017; originally announced March 2017.

Comments: To appear in IEEE Transactions on Information Theory

Active Ranking from Pairwise Comparisons and when Parametric Assumptions Don't Help

Authors: Reinhard Heckel, Nihar B. Shah, Kannan Ramchandran, Martin J. Wainwright

Abstract: We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items into sets of pre-specified sizes according to their scores. This notion of ranking includes as special cases the identification of the top-k items and the total or… ▽ More We consider sequential or active ranking of a set of n items based on noisy pairwise comparisons. Items are ranked according to the probability that a given item beats a randomly chosen item, and ranking refers to partitioning the items into sets of pre-specified sizes according to their scores. This notion of ranking includes as special cases the identification of the top-k items and the total ordering of the items. We first analyze a sequential ranking algorithm that counts the number of comparisons won, and uses these counts to decide whether to stop, or to compare another pair of items, chosen based on confidence intervals specified by the data collected up to that point. We prove that this algorithm succeeds in recovering the ranking using a number of comparisons that is optimal up to logarithmic factors. This guarantee does not require any structural properties of the underlying pairwise probability matrix, unlike a significant body of past work on pairwise ranking based on parametric models such as the Thurstone or Bradley-Terry-Luce models. It has been a long-standing open question as to whether or not imposing these parametric assumptions allows for improved ranking algorithms. For stochastic comparison models, in which the pairwise probabilities are bounded away from zero, our second contribution is to resolve this issue by proving a lower bound for parametric models. This shows, perhaps surprisingly, that these popular parametric modeling choices offer at most logarithmic gains for stochastic comparisons. △ Less

Submitted 23 September, 2016; v1 submitted 28 June, 2016; originally announced June 2016.

Comments: improved log factor in main result; added discussion on comparison probabilities close to zero; added numerical results

Fast and Robust Compressive Phase Retrieval with Sparse-Graph Codes

Authors: Dong Yin, Kangwook Lee, Ramtin Pedarsani, Kannan Ramchandran

Abstract: In this paper, we tackle the compressive phase retrieval problem in the presence of noise. The noisy compressive phase retrieval problem is to recover a $K$-sparse complex signal $s \in \mathbb{C}^n$, from a set of $m$ noisy quadratic measurements: $ y_i=| a_i^H s |^2+w_i$, where $a_i^H\in\mathbb{C}^n$ is the $i$th row of the measurement matrix $A\in\mathbb{C}^{m\times n}$, and $w_i$ is the additi… ▽ More In this paper, we tackle the compressive phase retrieval problem in the presence of noise. The noisy compressive phase retrieval problem is to recover a $K$-sparse complex signal $s \in \mathbb{C}^n$, from a set of $m$ noisy quadratic measurements: $ y_i=| a_i^H s |^2+w_i$, where $a_i^H\in\mathbb{C}^n$ is the $i$th row of the measurement matrix $A\in\mathbb{C}^{m\times n}$, and $w_i$ is the additive noise to the $i$th measurement. We consider the regime where $K=βn^δ$, with constants $β>0$ and $δ\in(0,1)$. We use the architecture of PhaseCode algorithm, and robustify it using two schemes: the almost-linear scheme and the sublinear scheme. We prove that with high probability, the almost-linear scheme recovers $s$ with sample complexity $Θ(K \log(n))$ and computational complexity $Θ(n \log(n))$, and the sublinear scheme recovers $s$ with sample complexity $Θ(K\log^3(n))$ and computational complexity $Θ(K\log^3(n))$. To the best of our knowledge, this is the first scheme that achieves sublinear computational complexity for compressive phase retrieval problem. Finally, we provide simulation results that support our theoretical contributions. △ Less

Submitted 1 June, 2016; originally announced June 2016.

CYCLADES: Conflict-free Asynchronous Machine Learning

Authors: Xinghao Pan, Maximilian Lam, Stephen Tu, Dimitris Papailiopoulos, Ce Zhang, Michael I. Jordan, Kannan Ramchandran, Chris Re, Benjamin Recht

Abstract: We present CYCLADES, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. CYCLADES is asynchronous during shared model updates, and requires no memory locking mechanisms, similar to HOGWILD!-type algorithms. Unlike HOGWILD!, CYCLADES introduces no conflicts during the parallel execution, and offers a black-box analysis for provable speedups across a… ▽ More We present CYCLADES, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. CYCLADES is asynchronous during shared model updates, and requires no memory locking mechanisms, similar to HOGWILD!-type algorithms. Unlike HOGWILD!, CYCLADES introduces no conflicts during the parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent conflict-free nature and cache locality, our multi-core implementation of CYCLADES consistently outperforms HOGWILD!-type algorithms on sufficiently sparse datasets, leading to up to 40% speedup gains compared to the HOGWILD! implementation of SGD, and up to 5x gains over asynchronous implementations of variance reduction algorithms. △ Less

Submitted 31 May, 2016; originally announced May 2016.

Speeding Up Distributed Machine Learning Using Codes

Authors: Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos, Kannan Ramchandran

Abstract: Codes are widely used in many engineering applications to offer robustness against noise. In large-scale systems there are several types of noise that can affect the performance of distributed machine learning algorithms -- straggler nodes, system failures, or communication bottlenecks -- but there has been little interaction cutting across codes, machine learning, and distributed systems. In this… ▽ More Codes are widely used in many engineering applications to offer robustness against noise. In large-scale systems there are several types of noise that can affect the performance of distributed machine learning algorithms -- straggler nodes, system failures, or communication bottlenecks -- but there has been little interaction cutting across codes, machine learning, and distributed systems. In this work, we provide theoretical insights on how coded solutions can achieve significant gains compared to uncoded ones. We focus on two of the most basic building blocks of distributed learning algorithms: matrix multiplication and data shuffling. For matrix multiplication, we use codes to alleviate the effect of stragglers, and show that if the number of homogeneous workers is $n$, and the runtime of each subtask has an exponential tail, coded computation can speed up distributed matrix multiplication by a factor of $\log n$. For data shuffling, we use codes to reduce communication bottlenecks, exploiting the excess in storage. We show that when a constant fraction $α$ of the data matrix can be cached at each worker, and $n$ is the number of workers, \emph{coded shuffling} reduces the communication cost by a factor of $(α+ \frac{1}{n})γ(n)$ compared to uncoded shuffling, where $γ(n)$ is the ratio of the cost of unicasting $n$ messages to $n$ users to multicasting a common message (of the same size) to $n$ users. For instance, $γ(n) \simeq n$ if multicasting a message to $n$ users is as cheap as unicasting a message to one user. We also provide experiment results, corroborating our theoretical gains of the coded algorithms. △ Less

Submitted 28 January, 2018; v1 submitted 8 December, 2015; originally announced December 2015.

Comments: This work is published in IEEE Transactions on Information Theory and presented in part at the NIPS 2015 Workshop on Machine Learning Systems and the IEEE ISIT 2016

Fast and Efficient Sparse 2D Discrete Fourier Transform using Sparse-Graph Codes

Authors: Frank Ong, Sameer Pawar, Kannan Ramchandran

Abstract: We present a novel algorithm, named the 2D-FFAST, to compute a sparse 2D-Discrete Fourier Transform (2D-DFT) featuring both low sample complexity and low computational complexity. The proposed algorithm is based on mixed concepts from signal processing (sub-sampling and aliasing), coding theory (sparse-graph codes) and number theory (Chinese-remainder-theorem) and generalizes the 1D-FFAST 2 algori… ▽ More We present a novel algorithm, named the 2D-FFAST, to compute a sparse 2D-Discrete Fourier Transform (2D-DFT) featuring both low sample complexity and low computational complexity. The proposed algorithm is based on mixed concepts from signal processing (sub-sampling and aliasing), coding theory (sparse-graph codes) and number theory (Chinese-remainder-theorem) and generalizes the 1D-FFAST 2 algorithm recently proposed by Pawar and Ramchandran [1] to the 2D setting. Concretely, our proposed 2D-FFAST algorithm computes a k-sparse 2D-DFT, with a uniformly random support, of size N = Nx x Ny using O(k) noiseless spatial-domain measurements in O(k log k) computational time. Our results are attractive when the sparsity is sub-linear with respect to the signal dimension, that is, when k -> infinity and k/N -> 0. For the case when the spatial-domain measurements are corrupted by additive noise, our 2D-FFAST framework extends to a noise-robust version in sub-linear time of O(k log4 N ) using O(k log3 N ) measurements. Simulation results, on synthetic images as well as real-world magnetic resonance images, are provided in Section VII and demonstrate the empirical performance of the proposed 2D-FFAST algorithm. △ Less

Submitted 19 September, 2015; originally announced September 2015.

Hiding the Rumor Source

Authors: Giulia Fanti, Peter Kairouz, Sewoong Oh, Kannan Ramchandran, Pramod Viswanath

Abstract: Anonymous social media platforms like Secret, Yik Yak, and Whisper have emerged as important tools for sharing ideas without the fear of judgment. Such anonymous platforms are also important in nations under authoritarian rule, where freedom of expression and the personal safety of message authors may depend on anonymity. Whether for fear of judgment or retribution, it is sometimes crucial to hide… ▽ More Anonymous social media platforms like Secret, Yik Yak, and Whisper have emerged as important tools for sharing ideas without the fear of judgment. Such anonymous platforms are also important in nations under authoritarian rule, where freedom of expression and the personal safety of message authors may depend on anonymity. Whether for fear of judgment or retribution, it is sometimes crucial to hide the identities of users who post sensitive messages. In this paper, we consider a global adversary who wishes to identify the author of a message; it observes either a snapshot of the spread of a message at a certain time, sampled timestamp metadata, or both. Recent advances in rumor source detection show that existing messaging protocols are vulnerable against such an adversary. We introduce a novel messaging protocol, which we call adaptive diffusion, and show that under the snapshot adversarial model, adaptive diffusion spreads content fast and achieves perfect obfuscation of the source when the underlying contact network is an infinite regular tree. That is, all users with the message are nearly equally likely to have been the origin of the message. When the contact network is an irregular tree, we characterize the probability of maximum likelihood detection by proving a concentration result over Galton-Watson trees. Experiments on a sampled Facebook network demonstrate that adaptive diffusion effectively hides the location of the source even when the graph is finite, irregular and has cycles. △ Less

Submitted 24 August, 2016; v1 submitted 9 September, 2015; originally announced September 2015.

Optimal Systematic Distributed Storage Codes with Fast Encoding

Authors: Preetum Nakkiran, K. V. Rashmi, Kannan Ramchandran

Abstract: Erasure codes are being increasingly used in distributed-storage systems in place of data-replication, since they provide the same level of reliability with much lower storage overhead. We consider the problem of constructing explicit erasure codes for distributed storage with the following desirable properties motivated by practice: (i) Maximum-Distance-Separable (MDS): to provide maximal reliabi… ▽ More Erasure codes are being increasingly used in distributed-storage systems in place of data-replication, since they provide the same level of reliability with much lower storage overhead. We consider the problem of constructing explicit erasure codes for distributed storage with the following desirable properties motivated by practice: (i) Maximum-Distance-Separable (MDS): to provide maximal reliability at minimum storage overhead, (ii) Optimal repair-bandwidth: to minimize the amount of data needed to be transferred to repair a failed node from remaining ones, (iii) Flexibility in repair: to allow maximal flexibility in selecting subset of nodes to use for repair, which includes not requiring that all surviving nodes be used for repair, (iv) Systematic Form: to ensure that the original data exists in uncoded form, and (v) Fast encoding: to minimize the cost of generating encoded data (enabled by a sparse generator matrix). This paper presents the first explicit code construction which theoretically guarantees all the five desired properties simultaneously. Our construction builds on a powerful class of codes called Product-Matrix (PM) codes. PM codes satisfy properties (i)-(iii), and either (iv) or (v), but not both simultaneously. Indeed, native PM codes have inherent structure that leads to sparsity, but this structure is destroyed when the codes are made systematic. We first present an analytical framework for understanding the interaction between the design of PM codes and the systematic property. Using this framework, we provide an explicit code construction that simultaneously achieves all the above desired properties. We also present general ways of transforming existing storage and repair optimal codes to enable fast encoding through sparsity. In practice, such sparse codes result in encoding speedup by a factor of about 4 for typical parameters. △ Less

Submitted 6 September, 2015; originally announced September 2015.

Comments: 16 pages, 4 figures

SPRIGHT: A Fast and Robust Framework for Sparse Walsh-Hadamard Transform

Authors: Xiao Li, Joseph K. Bradley, Sameer Pawar, Kannan Ramchandran

Abstract: We consider the problem of computing the Walsh-Hadamard Transform (WHT) of some $N$-length input vector in the presence of noise, where the $N$-point Walsh spectrum is $K$-sparse with $K = {O}(N^δ)$ scaling sub-linearly in the input dimension $N$ for some $0<δ<1$. Over the past decade, there has been a resurgence in research related to the computation of Discrete Fourier Transform (DFT) for some l… ▽ More We consider the problem of computing the Walsh-Hadamard Transform (WHT) of some $N$-length input vector in the presence of noise, where the $N$-point Walsh spectrum is $K$-sparse with $K = {O}(N^δ)$ scaling sub-linearly in the input dimension $N$ for some $0<δ<1$. Over the past decade, there has been a resurgence in research related to the computation of Discrete Fourier Transform (DFT) for some length-$N$ input signal that has a $K$-sparse Fourier spectrum. In particular, through a sparse-graph code design, our earlier work on the Fast Fourier Aliasing-based Sparse Transform (FFAST) algorithm computes the $K$-sparse DFT in time ${O}(K\log K)$ by taking ${O}(K)$ noiseless samples. Inspired by the coding-theoretic design framework, Scheibler et al. proposed the Sparse Fast Hadamard Transform (SparseFHT) algorithm that elegantly computes the $K$-sparse WHT in the absence of noise using ${O}(K\log N)$ samples in time ${O}(K\log^2 N)$. However, the SparseFHT algorithm explicitly exploits the noiseless nature of the problem, and is not equipped to deal with scenarios where the observations are corrupted by noise. Therefore, a question of critical interest is whether this coding-theoretic framework can be made robust to noise. Further, if the answer is yes, what is the extra price that needs to be paid for being robust to noise? In this paper, we show, quite interestingly, that there is {\it no extra price} that needs to be paid for being robust to noise other than a constant factor. In other words, we can maintain the same sample complexity ${O}(K\log N)$ and the computational complexity ${O}(K\log^2 N)$ as those of the noiseless case, using our SParse Robust Iterative Graph-based Hadamard Transform (SPRIGHT) algorithm. △ Less

Submitted 25 August, 2015; originally announced August 2015.

Comments: Part of our results was reported in ISIT 2014, titled "The SPRIGHT algorithm for robust sparse Hadamard Transforms."

SAFFRON: A Fast, Efficient, and Robust Framework for Group Testing based on Sparse-Graph Codes

Authors: Kangwook Lee, Ramtin Pedarsani, Kannan Ramchandran

Abstract: Group testing tackles the problem of identifying a population of $K$ defective items from a set of $n$ items by pooling groups of items efficiently in order to cut down the number of tests needed. The result of a test for a group of items is positive if any of the items in the group is defective and negative otherwise. The goal is to judiciously group subsets of items such that defective items can… ▽ More Group testing tackles the problem of identifying a population of $K$ defective items from a set of $n$ items by pooling groups of items efficiently in order to cut down the number of tests needed. The result of a test for a group of items is positive if any of the items in the group is defective and negative otherwise. The goal is to judiciously group subsets of items such that defective items can be reliably recovered using the minimum number of tests, while also having a low-complexity decoding procedure. We describe SAFFRON (Sparse-grAph codes Framework For gROup testiNg), a non-adaptive group testing paradigm that recovers at least a $(1-ε)$-fraction (for any arbitrarily small $ε> 0$) of $K$ defective items with high probability with $m=6C(ε)K\log_2{n}$ tests, where $C(ε)$ is a precisely characterized constant that depends only on $ε$. For instance, it can provably recover at least $(1-10^{-6})K$ defective items with $m \simeq 68 K \log_2{n}$ tests. The computational complexity of the decoding algorithm of SAFFRON is $\mathcal{O}(K\log n)$, which is order-optimal. Further, we robustify SAFFRON such that it can reliably recover the set of $K$ defective items even in the presence of erroneous or noisy test results. We also propose Singleton-Only-SAFFRON, a variant of SAFFRON, that recovers all the $K$ defective items with $m=2e(1+α)K\log K \log_2 n$ tests with probability $1-\mathcal{O}{\left(\frac{1}{K^α}\right)}$, where $α>0$ is a constant. By leveraging powerful design and analysis tools from modern sparse-graph coding theory, SAFFRON is the first approach to reliable, large-scale probabilistic group testing that offers both precisely characterizable number of tests needed (down to the constants) together with order-optimal decoding complexity. △ Less

Submitted 18 August, 2015; originally announced August 2015.

Information-theoretically Secure Erasure Codes for Distributed Storage

Authors: Nihar B. Shah, K. V. Rashmi, Kannan Ramchandran, P. Vijay Kumar

Abstract: Repair operations in distributed storage systems potentially expose the data to malicious acts of passive eavesdroppers or active adversaries, which can be detrimental to the security of the system. This paper presents erasure codes and repair algorithms that ensure security of the data in the presence of passive eavesdroppers and active adversaries, while maintaining high availability, reliabilit… ▽ More Repair operations in distributed storage systems potentially expose the data to malicious acts of passive eavesdroppers or active adversaries, which can be detrimental to the security of the system. This paper presents erasure codes and repair algorithms that ensure security of the data in the presence of passive eavesdroppers and active adversaries, while maintaining high availability, reliability and efficiency in the system. Our codes are optimal in that they meet previously proposed lower bounds on the storage, network-bandwidth, and reliability requirements for a wide range of system parameters. Our results thus establish the capacity of such systems. Our codes for security from active adversaries provide an additional appealing feature of `on-demand security' where the desired level of security can be chosen separately for each instance of repair, and our algorithms remain optimal simultaneously for all possible levels. The paper also provides necessary and sufficient conditions governing the transformation of any (non-secure) code into one providing on-demand security. △ Less

Submitted 15 August, 2015; originally announced August 2015.

Perturbed Iterate Analysis for Asynchronous Stochastic Optimization

Authors: Horia Mania, Xinghao Pan, Dimitris Papailiopoulos, Benjamin Recht, Kannan Ramchandran, Michael I. Jordan

Abstract: We introduce and analyze stochastic optimization methods where the input to each gradient update is perturbed by bounded noise. We show that this framework forms the basis of a unified approach to analyze asynchronous implementations of stochastic optimization algorithms.In this framework, asynchronous stochastic optimization algorithms can be thought of as serial methods operating on noisy inputs… ▽ More We introduce and analyze stochastic optimization methods where the input to each gradient update is perturbed by bounded noise. We show that this framework forms the basis of a unified approach to analyze asynchronous implementations of stochastic optimization algorithms.In this framework, asynchronous stochastic optimization algorithms can be thought of as serial methods operating on noisy inputs. Using our perturbed iterate framework, we provide new analyses of the Hogwild! algorithm and asynchronous stochastic coordinate descent, that are simpler than earlier analyses, remove many assumptions of previous models, and in some cases yield improved upper bounds on the convergence rates. We proceed to apply our framework to develop and analyze KroMagnon: a novel, parallel, sparse stochastic variance-reduced gradient (SVRG) algorithm. We demonstrate experimentally on a 16-core machine that the sparse and parallel version of SVRG is in some cases more than four orders of magnitude faster than the standard SVRG algorithm. △ Less

Submitted 25 March, 2016; v1 submitted 24 July, 2015; originally announced July 2015.

Comments: 30 pages

MSC Class: 65K10; 65Y05; 68W10; 68W20

Parallel Correlation Clustering on Big Graphs

Authors: Xinghao Pan, Dimitris Papailiopoulos, Samet Oymak, Benjamin Recht, Kannan Ramchandran, Michael I. Jordan

Abstract: Given a similarity graph between items, correlation clustering (CC) groups similar items together and dissimilar ones apart. One of the most popular CC algorithms is KwikCluster: an algorithm that serially clusters neighborhoods of vertices, and obtains a 3-approximation ratio. Unfortunately, KwikCluster in practice requires a large number of clustering rounds, a potential bottleneck for large gra… ▽ More Given a similarity graph between items, correlation clustering (CC) groups similar items together and dissimilar ones apart. One of the most popular CC algorithms is KwikCluster: an algorithm that serially clusters neighborhoods of vertices, and obtains a 3-approximation ratio. Unfortunately, KwikCluster in practice requires a large number of clustering rounds, a potential bottleneck for large graphs. We present C4 and ClusterWild!, two algorithms for parallel correlation clustering that run in a polylogarithmic number of rounds and achieve nearly linear speedups, provably. C4 uses concurrency control to enforce serializability of a parallel clustering process, and guarantees a 3-approximation ratio. ClusterWild! is a coordination free algorithm that abandons consistency for the benefit of better scaling; this leads to a provably small loss in the 3-approximation ratio. We provide extensive experimental results for both algorithms, where we outperform the state of the art, both in terms of clustering accuracy and running time. We show that our algorithms can cluster billion-edge graphs in under 5 seconds on 32 cores, while achieving a 15x speedup. △ Less

Submitted 20 July, 2015; v1 submitted 17 July, 2015; originally announced July 2015.

Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependence

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, Martin J. Wainwright

Abstract: Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others. We consider parametric ordinal models for such pairwise comparison data involving a latent vector $w^* \in \mathbb{R}^d$ that represents the "qualities" of the $d$ items being compared; this class of models includes the two most widely used parame… ▽ More Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others. We consider parametric ordinal models for such pairwise comparison data involving a latent vector $w^* \in \mathbb{R}^d$ that represents the "qualities" of the $d$ items being compared; this class of models includes the two most widely used parametric models--the Bradley-Terry-Luce (BTL) and the Thurstone models. Working within a standard minimax framework, we provide tight upper and lower bounds on the optimal error in estimating the quality score vector $w^*$ under this class of models. The bounds depend on the topology of the comparison graph induced by the subset of pairs being compared via its Laplacian spectrum. Thus, in settings where the subset of pairs may be chosen, our results provide principled guidelines for making this choice. Finally, we compare these error rates to those under cardinal measurement models and show that the error rates in the ordinal and cardinal settings have identical scalings apart from constant pre-factors. △ Less

Submitted 6 May, 2015; originally announced May 2015.

Comments: 39 pages, 5 figures. Significant extension of arXiv:1406.6618

Efficient Algorithms for the Data Exchange Problem

Authors: Nebojsa Milosavljevic, Sameer Pawar, Salim El Rouayheb, Michael Gastpar, Kannan Ramchandran

Abstract: In this paper we study the data exchange problem where a set of users is interested in gaining access to a common file, but where each has only partial knowledge about it as side-information. Assuming that the file is broken into packets, the side-information considered is in the form of linear combinations of the file packets. Given that the collective information of all the users is sufficient t… ▽ More In this paper we study the data exchange problem where a set of users is interested in gaining access to a common file, but where each has only partial knowledge about it as side-information. Assuming that the file is broken into packets, the side-information considered is in the form of linear combinations of the file packets. Given that the collective information of all the users is sufficient to allow recovery of the entire file, the goal is for each user to gain access to the file while minimizing some communication cost. We assume that users can communicate over a noiseless broadcast channel, and that the communication cost is a sum of each user's cost function over the number of bits it transmits. For instance, the communication cost could simply be the total number of bits that needs to be transmitted. In the most general case studied in this paper, each user can have any arbitrary convex cost function. We provide deterministic, polynomial-time algorithms (in the number of users and packets) which find an optimal communication scheme that minimizes the communication cost. To further lower the complexity, we also propose a simple randomized algorithm inspired by our deterministic algorithm which is based on a random linear network coding scheme. △ Less

Submitted 21 February, 2015; originally announced February 2015.

Comments: submitted to Transactions on Information Theory

A robust sub-linear time R-FFAST algorithm for computing a sparse DFT

Authors: Sameer Pawar, Kannan Ramchandran

Abstract: The Fast Fourier Transform (FFT) is the most efficiently known way to compute the Discrete Fourier Transform (DFT) of an arbitrary n-length signal, and has a computational complexity of O(n log n). If the DFT X of the signal x has only k non-zero coefficients (where k < n), can we do better? In [1], we addressed this question and presented a novel FFAST (Fast Fourier Aliasing-based Sparse Transfor… ▽ More The Fast Fourier Transform (FFT) is the most efficiently known way to compute the Discrete Fourier Transform (DFT) of an arbitrary n-length signal, and has a computational complexity of O(n log n). If the DFT X of the signal x has only k non-zero coefficients (where k < n), can we do better? In [1], we addressed this question and presented a novel FFAST (Fast Fourier Aliasing-based Sparse Transform) algorithm that cleverly induces sparse graph alias codes in the DFT domain, via a Chinese-Remainder-Theorem (CRT)-guided sub-sampling operation of the time-domain samples. The resulting sparse graph alias codes are then exploited to devise a fast and iterative onion-peeling style decoder that computes an n length DFT of a signal using only O(k) time-domain samples and O(klog k) computations. The FFAST algorithm is applicable whenever k is sub-linear in n (i.e. k = o(n)), but is obviously most attractive when k is much smaller than n. In this paper, we adapt the FFAST framework of [1] to the case where the time-domain samples are corrupted by a white Gaussian noise. In particular, we show that the extended noise robust algorithm R-FFAST computes an n-length k-sparse DFT X using O(klog ^3 n) noise-corrupted time-domain samples, in O(klog^4n) computations, i.e., sub-linear time complexity. While our theoretical results are for signals with a uniformly random support of the non-zero DFT coefficients and additive white Gaussian noise, we provide simulation results which demonstrates that the R-FFAST algorithm performs well even for signals like MR images, that have an approximately sparse Fourier spectrum with a non-uniform support for the dominant DFT coefficients. △ Less

Submitted 1 January, 2015; originally announced January 2015.

Comments: 35 pages, 7 figures

Sub-linear Time Support Recovery for Compressed Sensing using Sparse-Graph Codes

Authors: Xiao Li, Dong Yin, Sameer Pawar, Ramtin Pedarsani, Kannan Ramchandran

Abstract: We study the support recovery problem for compressed sensing, where the goal is to reconstruct the a high-dimensional $K$-sparse signal $\mathbf{x}\in\mathbb{R}^N$, from low-dimensional linear measurements with and without noise. Our key contribution is a new compressed sensing framework through a new family of carefully designed sparse measurement matrices associated with minimal measurement cost… ▽ More We study the support recovery problem for compressed sensing, where the goal is to reconstruct the a high-dimensional $K$-sparse signal $\mathbf{x}\in\mathbb{R}^N$, from low-dimensional linear measurements with and without noise. Our key contribution is a new compressed sensing framework through a new family of carefully designed sparse measurement matrices associated with minimal measurement costs and a low-complexity recovery algorithm. The measurement matrix in our framework is designed based on the well-crafted sparsification through capacity-approaching sparse-graph codes, where the sparse coefficients can be recovered efficiently in a few iterations by performing simple error decoding over the observations. We formally connect this general recovery problem with sparse-graph decoding in packet communication systems, and analyze our framework in terms of the measurement cost, time complexity and recovery performance. In the noiseless setting, our framework can recover any arbitrary $K$-sparse signal in $O(K)$ time using $2K$ measurements asymptotically with high probability. In the noisy setting, when the sparse coefficients take values in a finite and quantized alphabet, our framework can achieve the same goal in time $O(K\log(N/K))$ using $O(K\log(N/K))$ measurements obtained from measurement matrix with elements $\{-1,0,1\}$. When the sparsity $K$ is sub-linear in the signal dimension $K=O(N^δ)$ for some $0<δ<1$, our results are order-optimal in terms of measurement costs and run-time, both of which are sub-linear in the signal dimension $N$. The sub-linear measurement cost and run-time can also be achieved with continuous-valued sparse coefficients, with a slight increment in the logarithmic factors. This offers the desired scalability of our framework that can potentially enable real-time or near-real-time processing for massive datasets featuring sparsity. △ Less

Submitted 25 February, 2018; v1 submitted 24 December, 2014; originally announced December 2014.

Capacity-Approaching PhaseCode for Low-Complexity Compressive Phase Retrieval

Authors: Ramtin Pedarsani, Kangwook Lee, Kannan Ramchandran

Abstract: In this paper, we tackle the general compressive phase retrieval problem. The problem is to recover a K-sparse complex vector of length n, $x\in \mathbb{C}^n$, from the magnitudes of m linear measurements, $y=|Ax|$, where $A \in \mathbb{C}^{m \times n}$ can be designed, and the magnitudes are taken component-wise for vector $Ax\in \mathbb{C}^m$. We propose a variant of the PhaseCode algorithm, and… ▽ More In this paper, we tackle the general compressive phase retrieval problem. The problem is to recover a K-sparse complex vector of length n, $x\in \mathbb{C}^n$, from the magnitudes of m linear measurements, $y=|Ax|$, where $A \in \mathbb{C}^{m \times n}$ can be designed, and the magnitudes are taken component-wise for vector $Ax\in \mathbb{C}^m$. We propose a variant of the PhaseCode algorithm, and show that, using an irregular left-degree sparse-graph code construction, the algorithm can recover almost all the K non-zero signal components using only slightly more than 4K measurements under some mild assumptions, with optimal time and memory complexity of ${\cal O}(K)$. It is known that the fundamental limit for the number of measurements in compressive phase retrieval problem is $4K - o(K)$. To the best of our knowledge, this is the first constructive capacity-approaching compressive phase retrieval algorithm. As a second contribution, we propose another variant of the PhaseCode algorithm that is based on a Compressive Sensing framework involving sparse-graph codes. Our proposed algorithm is an instance of a more powerful "separation" architecture that can be used to address the compressive phase-retrieval problem in general. This modular design features a compressive sensing outer layer, and a trigonometric-based phase-retrieval inner layer. The compressive-sensing layer operates as a linear phase-aware compressive measurement subsystem, while the trig-based phase-retrieval layer provides the desired abstraction between the actually targeted nonlinear phase-retrieval problem and the induced linear compressive-sensing problem. Invoking this architecture based on the use of sparse-graph codes for the compressive sensing layer, we show that we can exactly recover a signal from only the magnitudes of its linear measurements using only slightly more than 6K measurements. △ Less

Submitted 16 February, 2015; v1 submitted 17 December, 2014; originally announced December 2014.

Comments: arXiv admin note: text overlap with arXiv:1408.0034

PhaseCode: Fast and Efficient Compressive Phase Retrieval based on Sparse-Graph-Codes

Authors: Ramtin Pedarsani, Dong Yin, Kangwook Lee, Kannan Ramchandran

Abstract: We consider the problem of recovering a $K$-sparse complex signal $x$ from $m$ intensity measurements. We propose the PhaseCode algorithm, and show that in the noiseless case, PhaseCode can recover an arbitrarily-close-to-one fraction of the $K$ non-zero signal components using only slightly more than $4K$ measurements when the support of the signal is uniformly random, with order-optimal time and… ▽ More We consider the problem of recovering a $K$-sparse complex signal $x$ from $m$ intensity measurements. We propose the PhaseCode algorithm, and show that in the noiseless case, PhaseCode can recover an arbitrarily-close-to-one fraction of the $K$ non-zero signal components using only slightly more than $4K$ measurements when the support of the signal is uniformly random, with order-optimal time and memory complexity of $Θ(K)$. It is known that the fundamental limit for the number of measurements in compressive phase retrieval problem is $4K - o(K)$ to recover the signal exactly and with no assumptions on its support distribution. This shows that under mild relaxation of the conditions, our algorithm is the first constructive \emph{capacity-approaching} compressive phase retrieval algorithm: in fact, our algorithm is also order-optimal in complexity and memory. Next, motivated by some important practical classes of optical systems, we consider a Fourier-friendly constrained measurement setting, and show that its performance matches that of the unconstrained setting. In the Fourier-friendly setting that we consider, the measurement matrix is constrained to be a cascade of Fourier matrices and diagonal matrices. We further demonstrate how PhaseCode can be robustified to noise. Throughout, we provide extensive simulation results that validate the practical power of our proposed algorithms for the sparse unconstrained and Fourier-friendly measurement settings, for noiseless and noisy scenarios. A key contribution of our work is the novel use of coding-theoretic tools like density evolution methods for the design and analysis of fast and efficient algorithms for compressive phase-retrieval problems. △ Less

Submitted 30 March, 2017; v1 submitted 31 July, 2014; originally announced August 2014.

Comments: To appear in IEEE Transactions on Information Theory

When is it Better to Compare than to Score?

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, Martin Wainwright

Abstract: When eliciting judgements from humans for an unknown quantity, one often has the choice of making direct-scoring (cardinal) or comparative (ordinal) measurements. In this paper we study the relative merits of either choice, providing empirical and theoretical guidelines for the selection of a measurement scheme. We provide empirical evidence based on experiments on Amazon Mechanical Turk that in a… ▽ More When eliciting judgements from humans for an unknown quantity, one often has the choice of making direct-scoring (cardinal) or comparative (ordinal) measurements. In this paper we study the relative merits of either choice, providing empirical and theoretical guidelines for the selection of a measurement scheme. We provide empirical evidence based on experiments on Amazon Mechanical Turk that in a variety of tasks, (pairwise-comparative) ordinal measurements have lower per sample noise and are typically faster to elicit than cardinal ones. Ordinal measurements however typically provide less information. We then consider the popular Thurstone and Bradley-Terry-Luce (BTL) models for ordinal measurements and characterize the minimax error rates for estimating the unknown quantity. We compare these minimax error rates to those under cardinal measurement models and quantify for what noise levels ordinal measurements are better. Finally, we revisit the data collected from our experiments and show that fitting these models confirms this prediction: for tasks where the noise in ordinal measurements is sufficiently low, the ordinal approach results in smaller errors in the estimation. △ Less

Submitted 25 June, 2014; originally announced June 2014.

When Do Redundant Requests Reduce Latency ?

Authors: Nihar B. Shah, Kangwook Lee, Kannan Ramchandran

Abstract: Several systems possess the flexibility to serve requests in more than one way. For instance, a distributed storage system storing multiple replicas of the data can serve a request from any of the multiple servers that store the requested data, or a computational task may be performed in a compute-cluster by any one of multiple processors. In such systems, the latency of serving the requests may p… ▽ More Several systems possess the flexibility to serve requests in more than one way. For instance, a distributed storage system storing multiple replicas of the data can serve a request from any of the multiple servers that store the requested data, or a computational task may be performed in a compute-cluster by any one of multiple processors. In such systems, the latency of serving the requests may potentially be reduced by sending "redundant requests": a request may be sent to more servers than needed, and it is deemed served when the requisite number of servers complete service. Such a mechanism trades off the possibility of faster execution of at least one copy of the request with the increase in the delay due to an increased load on the system. Due to this tradeoff, it is unclear when redundant requests may actually help. Several recent works empirically evaluate the latency performance of redundant requests in diverse settings. This work aims at an analytical study of the latency performance of redundant requests, with the primary goals of characterizing under what scenarios sending redundant requests will help (and under what scenarios they will not help), as well as designing optimal redundant-requesting policies. We first present a model that captures the key features of such systems. We show that when service times are i.i.d. memoryless or "heavier", and when the additional copies of already-completed jobs can be removed instantly, redundant requests reduce the average latency. On the other hand, when service times are "lighter" or when service times are memoryless and removal of jobs is not instantaneous, then not having any redundancy in the requests is optimal under high loads. Our results hold for arbitrary arrival processes. △ Less

Submitted 6 November, 2013; originally announced November 2013.

Comments: Extended version of paper presented at Allerton Conference 2013

Low-Complexity Interactive Algorithms for Synchronization from Deletions, Insertions, and Substitutions

Authors: Ramji Venkataramanan, Vasuki Narasimha Swamy, Kannan Ramchandran

Abstract: Consider two remote nodes having binary sequences $X$ and $Y$, respectively. $Y$ is an edited version of ${X}$, where the editing involves random deletions, insertions, and substitutions, possibly in bursts. The goal is for the node with $Y$ to reconstruct $X$ with minimal exchange of information over a noiseless link. The communication is measured in terms of both the total number of bits exchang… ▽ More Consider two remote nodes having binary sequences $X$ and $Y$, respectively. $Y$ is an edited version of ${X}$, where the editing involves random deletions, insertions, and substitutions, possibly in bursts. The goal is for the node with $Y$ to reconstruct $X$ with minimal exchange of information over a noiseless link. The communication is measured in terms of both the total number of bits exchanged and the number of interactive rounds of communication. This paper focuses on the setting where the number of edits is $o(\tfrac{n}{\log n})$, where $n$ is the length of $X$. We first consider the case where the edits are a mixture of insertions and deletions (indels), and propose an interactive synchronization algorithm with near-optimal communication rate and average computational complexity of $O(n)$ arithmetic operations. The algorithm uses interaction to efficiently split the source sequence into substrings containing exactly one deletion or insertion. Each of these substrings is then synchronized using an optimal one-way synchronization code based on the single-deletion correcting channel codes of Varshamov and Tenengolts (VT codes). We then build on this synchronization algorithm in three different ways. First, it is modified to work with a single round of interaction. The reduction in the number of rounds comes at the expense of higher communication, which is quantified. Next, we present an extension to the practically important case where the insertions and deletions may occur in (potentially large) bursts. Finally, we show how to synchronize the sources to within a target Hamming distance. This feature can be used to differentiate between substitution and indel edits. In addition to theoretical performance bounds, we provide several validating simulation results for the proposed algorithms. △ Less

Submitted 12 September, 2015; v1 submitted 8 October, 2013; originally announced October 2013.

Journal ref: IEEE Transactions on Information Theory, vol. 61, no. 10, pp. 5670-5689, October 2015

A Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster

Authors: K. V. Rashmi, Nihar B. Shah, Dikang Gu, Hairong Kuang, Dhruba Borthakur, Kannan Ramchandran

Abstract: Erasure codes, such as Reed-Solomon (RS) codes, are being increasingly employed in data centers to combat the cost of reliably storing large amounts of data. Although these codes provide optimal storage efficiency, they require significantly high network and disk usage during recovery of missing data. In this paper, we first present a study on the impact of recovery operations of erasure-coded dat… ▽ More Erasure codes, such as Reed-Solomon (RS) codes, are being increasingly employed in data centers to combat the cost of reliably storing large amounts of data. Although these codes provide optimal storage efficiency, they require significantly high network and disk usage during recovery of missing data. In this paper, we first present a study on the impact of recovery operations of erasure-coded data on the data-center network, based on measurements from Facebook's warehouse cluster in production. To the best of our knowledge, this is the first study of its kind available in the literature. Our study reveals that recovery of RS-coded data results in a significant increase in network traffic, more than a hundred terabytes per day, in a cluster storing multiple petabytes of RS-coded data. To address this issue, we present a new storage code using our recently proposed "Piggybacking" framework, that reduces the network and disk usage during recovery by 30% in theory, while also being storage optimal and supporting arbitrary design parameters. The implementation of the proposed code in the Hadoop Distributed File System (HDFS) is underway. We use the measurements from the warehouse cluster to show that the proposed code would lead to a reduction of close to fifty terabytes of cross-rack traffic per day. △ Less

Submitted 1 September, 2013; originally announced September 2013.

Comments: In proceedings of USENIX HotStorage, San Jose, June 2013

Computing a k-sparse n-length Discrete Fourier Transform using at most 4k samples and O(k log k) complexity

Authors: Sameer Pawar, Kannan Ramchandran

Abstract: Given an $n$-length input signal $\mbf{x}$, it is well known that its Discrete Fourier Transform (DFT), $\mbf{X}$, can be computed in $O(n \log n)$ complexity using a Fast Fourier Transform (FFT). If the spectrum $\mbf{X}$ is exactly $k$-sparse (where $k<<n$), can we do better? We show that asymptotically in $k$ and $n$, when $k$ is sub-linear in $n$ (precisely, $k \propto n^δ$ where $0 < δ<1$), a… ▽ More Given an $n$-length input signal $\mbf{x}$, it is well known that its Discrete Fourier Transform (DFT), $\mbf{X}$, can be computed in $O(n \log n)$ complexity using a Fast Fourier Transform (FFT). If the spectrum $\mbf{X}$ is exactly $k$-sparse (where $k<<n$), can we do better? We show that asymptotically in $k$ and $n$, when $k$ is sub-linear in $n$ (precisely, $k \propto n^δ$ where $0 < δ<1$), and the support of the non-zero DFT coefficients is uniformly random, we can exploit this sparsity in two fundamental ways (i) {\bf {sample complexity}}: we need only $M=rk$ deterministically chosen samples of the input signal $\mbf{x}$ (where $r < 4$ when $0 < δ< 0.99$); and (ii) {\bf {computational complexity}}: we can reliably compute the DFT $\mbf{X}$ using $O(k \log k)$ operations, where the constants in the big Oh are small and are related to the constants involved in computing a small number of DFTs of length approximately equal to the sparsity parameter $k$. Our algorithm succeeds with high probability, with the probability of failure vanishing to zero asymptotically in the number of samples acquired, $M$. △ Less

Submitted 26 January, 2015; v1 submitted 3 May, 2013; originally announced May 2013.

Comments: 36 pages, 15 figures. To be presented at ISIT-2013, Istanbul Turkey

Optimal DNA shotgun sequencing: Noisy reads are as good as noiseless reads

Authors: Abolfazl Motahari, Kannan Ramchandran, David Tse, Nan Ma

Abstract: We establish the fundamental limits of DNA shotgun sequencing under noisy reads. We show a surprising result: for the i.i.d. DNA model, noisy reads are as good as noiseless reads, provided that the noise level is below a certain threshold which can be surprisingly high. As an example, for a uniformly distributed DNA sequence and a symmetric substitution noisy read channel, the threshold is as high… ▽ More We establish the fundamental limits of DNA shotgun sequencing under noisy reads. We show a surprising result: for the i.i.d. DNA model, noisy reads are as good as noiseless reads, provided that the noise level is below a certain threshold which can be surprisingly high. As an example, for a uniformly distributed DNA sequence and a symmetric substitution noisy read channel, the threshold is as high as 19%. △ Less

Submitted 9 April, 2013; originally announced April 2013.

Comments: Submitted to ISIT 2013

A Piggybacking Design Framework for Read-and Download-efficient Distributed Storage Codes

Authors: K. V. Rashmi, Nihar B. Shah, Kannan Ramchandran

Abstract: We present a new 'piggybacking' framework for designing distributed storage codes that are efficient in data-read and download required during node-repair. We illustrate the power of this framework by constructing classes of explicit codes that entail the smallest data-read and download for repair among all existing solutions for three important settings: (a) codes meeting the constraints of being… ▽ More We present a new 'piggybacking' framework for designing distributed storage codes that are efficient in data-read and download required during node-repair. We illustrate the power of this framework by constructing classes of explicit codes that entail the smallest data-read and download for repair among all existing solutions for three important settings: (a) codes meeting the constraints of being Maximum-Distance-Separable (MDS), high-rate and having a small number of substripes, arising out of practical considerations for implementation in data centers, (b) binary MDS codes for all parameters where binary MDS codes exist, (c) MDS codes with the smallest repair-locality. In addition, we employ this framework to enable efficient repair of parity nodes in existing codes that were originally constructed to address the repair of only the systematic nodes. The basic idea behind our framework is to take multiple instances of existing codes and add carefully designed functions of the data of one instance to the other. Typical savings in data-read during repair is 25% to 50% depending on the choice of the code parameters. △ Less

Submitted 24 February, 2013; originally announced February 2013.

Comments: Extended version of ISIT 2013 submission

The MDS Queue: Analysing the Latency Performance of Erasure Codes

Authors: Nihar B. Shah, Kangwook Lee, Kannan Ramchandran

Abstract: In order to scale economically, data centers are increasingly evolving their data storage methods from the use of simple data replication to the use of more powerful erasure codes, which provide the same level of reliability as replication but at a significantly lower storage cost. In particular, it is well known that Maximum-Distance-Separable (MDS) codes, such as Reed-Solomon codes, provide the… ▽ More In order to scale economically, data centers are increasingly evolving their data storage methods from the use of simple data replication to the use of more powerful erasure codes, which provide the same level of reliability as replication but at a significantly lower storage cost. In particular, it is well known that Maximum-Distance-Separable (MDS) codes, such as Reed-Solomon codes, provide the maximum storage efficiency. While the use of codes for providing improved reliability in archival storage systems, where the data is less frequently accessed (or so-called "cold data"), is well understood, the role of codes in the storage of more frequently accessed and active "hot data", where latency is the key metric, is less clear. In this paper, we study data storage systems based on MDS codes through the lens of queueing theory, and term this the "MDS queue." We analytically characterize the (average) latency performance of MDS queues, for which we present insightful scheduling policies that form upper and lower bounds to performance, and are observed to be quite tight. Extensive simulations are also provided and used to validate our theoretical analysis. We also employ the framework of the MDS queue to analyse different methods of performing so-called degraded reads (reading of partial data) in distributed data storage. △ Less

Submitted 10 November, 2013; v1 submitted 22 November, 2012; originally announced November 2012.

Semi-Definite Programming Relaxation for Non-Line-of-Sight Localization

Authors: Venkatesan Ekambaram, Giulia Fanti, Kannan Ramchandran

Abstract: We consider the problem of estimating the locations of a set of points in a k-dimensional euclidean space given a subset of the pairwise distance measurements between the points. We focus on the case when some fraction of these measurements can be arbitrarily corrupted by large additive noise. Given that the problem is highly non-convex, we propose a simple semidefinite programming relaxation that… ▽ More We consider the problem of estimating the locations of a set of points in a k-dimensional euclidean space given a subset of the pairwise distance measurements between the points. We focus on the case when some fraction of these measurements can be arbitrarily corrupted by large additive noise. Given that the problem is highly non-convex, we propose a simple semidefinite programming relaxation that can be efficiently solved using standard algorithms. We define a notion of non-contractibility and show that the relaxation gives the exact point locations when the underlying graph is non-contractible. The performance of the algorithm is evaluated on an experimental data set obtained from a network of 44 nodes in an indoor environment and is shown to be robust to non-line-of-sight errors. △ Less

Submitted 18 October, 2012; originally announced October 2012.

Distributed Secret Dissemination Across a Network

Authors: Nihar B. Shah, K. V. Rashmi, Kannan Ramchandran

Abstract: Shamir's (n, k) threshold secret sharing is an important component of several cryptographic protocols, such as those for secure multiparty-computation and key management. These protocols typically assume the presence of direct communication links from the dealer to all participants, in which case the dealer can directly pass the shares of the secret to each participant. In this paper, we consider… ▽ More Shamir's (n, k) threshold secret sharing is an important component of several cryptographic protocols, such as those for secure multiparty-computation and key management. These protocols typically assume the presence of direct communication links from the dealer to all participants, in which case the dealer can directly pass the shares of the secret to each participant. In this paper, we consider the problem of secret sharing when the dealer does not have direct communication links to all the participants, and instead, the dealer and the participants form a general network. Existing methods are based on secure message transmissions from the dealer to each participant requiring considerable coordination in the network. In this paper, we present a distributed algorithm for disseminating shares over a network, which we call the SNEAK algorithm, requiring each node to know only the identities of its one-hop neighbours. While SNEAK imposes a stronger condition on the network by requiring the dealer to be what we call k-propagating rather than k-connected as required by the existing solutions, we show that in addition to being distributed, SNEAK achieves significant reduction in the communication cost and the amount of randomness required. △ Less

Submitted 22 October, 2014; v1 submitted 30 June, 2012; originally announced July 2012.

Comments: Extended version of a paper presented at the International Symposium on Information Theory (ISIT) 2013

Collaborative High Accuracy Localization in Mobile Multipath Environments

Authors: Venkatesan. N. Ekambaram, Kannan Ramchandran, Raja Sengupta

Abstract: We study the problem of high accuracy localization of mobile nodes in a multipath-rich environment where sub-meter accuracies are required. We employ a peer-to-peer framework where the vehicles/nodes can get pairwise multipath-degraded ranging estimates in local neighborhoods together with a fixed number of anchor nodes. The challenge is to overcome the multipath-barrier with redundancy in order t… ▽ More We study the problem of high accuracy localization of mobile nodes in a multipath-rich environment where sub-meter accuracies are required. We employ a peer-to-peer framework where the vehicles/nodes can get pairwise multipath-degraded ranging estimates in local neighborhoods together with a fixed number of anchor nodes. The challenge is to overcome the multipath-barrier with redundancy in order to provide the desired accuracies especially under severe multipath conditions when the fraction of received signals corrupted by multipath is dominating. We invoke a analytical graphical model framework based on particle filtering and reveal its high accuracy localization promise through simulations. We also address design questions such as "How many anchors and what fraction of line-of-sight (LOS) measurements are needed to achieve a specified target accuracy?", by analytically characterizing the performance improvement in localization accuracy as a function of the number of nodes in the network and the fraction of LOS measurements. In particular, for a static node placement, we show that the Cramer-Rao Lower Bound (CRLB), a fundamental lower bound on the localization accuracy, can be expressed as a product of two factors - a scalar function that depends only on the parameters of the noise distribution and a matrix that depends only on the geometry of node locations and the underlying connectivity graph. Further, a simplified expression is obtained for the CRLB that helps deduce the scaling behavior of the estimation error as a function of the number of agents and anchors in the network. The bound suggests that even a small fraction of LOS measurements can provide significant improvements. Conversely, a small fraction of NLOS measurements can significantly degrade the performance. The analysis is extended to the mobile setting and the performance is compared with the derived CRLB. △ Less

Submitted 7 November, 2012; v1 submitted 13 June, 2012; originally announced June 2012.

Optimal Demand Response with Energy Storage Management

Authors: Longbo Huang, Jean Walrand, Kannan Ramchandran

Abstract: In this paper, we consider the problem of optimal demand response and energy storage management for a power consuming entity. The entity's objective is to find an optimal control policy for deciding how much load to consume, how much power to purchase from/sell to the power grid, and how to use the finite capacity energy storage device and renewable energy, to minimize his average cost, being the… ▽ More In this paper, we consider the problem of optimal demand response and energy storage management for a power consuming entity. The entity's objective is to find an optimal control policy for deciding how much load to consume, how much power to purchase from/sell to the power grid, and how to use the finite capacity energy storage device and renewable energy, to minimize his average cost, being the disutility due to load- shedding and cost for purchasing power. Due to the coupling effect of the finite size energy storage, such problems are challenging and are typically tackled using dynamic programming, which is often complex in computation and requires substantial statistical information of the system dynamics. We instead develop a low-complexity algorithm called Demand Response with Energy Storage Management (DR-ESM). DR-ESM does not require any statistical knowledge of the system dynamics, including the renewable energy and the power prices. It only requires the entity to solve a small convex optimization program with 6 variables and 6 linear constraints every time for decision making. We prove that DR-ESM is able to achieve near-optimal performance and explicitly compute the required energy storage size. △ Less

Submitted 19 May, 2012; originally announced May 2012.

Minimum Cost Multicast with Decentralized Sources

Authors: Nebojsa Milosavljevic, Sameer Pawar, Salim El Rouayheb, Michael Gastpar, Kannan Ramchandran

Abstract: In this paper we study the multisource multicast problem where every sink in a given directed acyclic graph is a client and is interested in a common file. We consider the case where each node can have partial knowledge about the file as a side information. Assuming that nodes can communicate over the capacity constrained links of the graph, the goal is for each client to gain access to the file,… ▽ More In this paper we study the multisource multicast problem where every sink in a given directed acyclic graph is a client and is interested in a common file. We consider the case where each node can have partial knowledge about the file as a side information. Assuming that nodes can communicate over the capacity constrained links of the graph, the goal is for each client to gain access to the file, while minimizing some linear cost function of number of bits transmitted in the network. We consider three types of side-information settings:(ii) side information in the form of linearly correlated packets; and (iii) the general setting where the side information at the nodes have an arbitrary (i.i.d.) correlation structure. In this work we 1) provide a polynomial time feasibility test, i.e., whether or not all the clients can recover the file, and 2) we provide a polynomial-time algorithm that finds the optimal rate allocation among the links of the graph, and then determines an explicit transmission scheme for cases (i) and (ii). △ Less

Submitted 10 March, 2012; originally announced March 2012.

Data Exchange Problem with Helpers

Authors: Nebojsa Milosavljevic, Sameer Pawar, Salim El Rouayheb, Michael Gastpar, Kannan Ramchandran

Abstract: In this paper we construct a deterministic polynomial time algorithm for the problem where a set of users is interested in gaining access to a common file, but where each has only partial knowledge of the file. We further assume the existence of another set of terminals in the system, called helpers, who are not interested in the common file, but who are willing to help the users. Given that the c… ▽ More In this paper we construct a deterministic polynomial time algorithm for the problem where a set of users is interested in gaining access to a common file, but where each has only partial knowledge of the file. We further assume the existence of another set of terminals in the system, called helpers, who are not interested in the common file, but who are willing to help the users. Given that the collective information of all the terminals is sufficient to allow recovery of the entire file, the goal is to minimize the (weighted) sum of bits that these terminals need to exchange over a noiseless public channel in order achieve this goal. Based on established connections to the multi-terminal secrecy problem, our algorithm also implies a polynomial-time method for constructing the largest shared secret key in the presence of an eavesdropper. We consider the following side-information settings: (i) side-information in the form of uncoded packets of the file, where the terminals' side-information consists of subsets of the file; (ii) side-information in the form of linearly correlated packets, where the terminals have access to linear combinations of the file packets; and (iii) the general setting where the the terminals' side-information has an arbitrary (i.i.d.) correlation structure. We provide a polynomial-time algorithm (in the number of terminals) that finds the optimal rate allocations for these terminals, and then determines an explicit optimal transmission scheme for cases (i) and (ii). △ Less

Submitted 8 February, 2012; originally announced February 2012.

Codes Can Reduce Queueing Delay in Data Centers

Authors: Longbo Huang, Sameer Pawar, Hao Zhang, Kannan Ramchandran

Abstract: In this paper, we quantify how much codes can reduce the data retrieval latency in storage systems. By combining a simple linear code with a novel request scheduling algorithm, which we call Blocking-one Scheduling (BoS), we show analytically that it is possible to reduce data retrieval delay by up to 17% over currently popular replication-based strategies. Although in this work we focus on a simp… ▽ More In this paper, we quantify how much codes can reduce the data retrieval latency in storage systems. By combining a simple linear code with a novel request scheduling algorithm, which we call Blocking-one Scheduling (BoS), we show analytically that it is possible to reduce data retrieval delay by up to 17% over currently popular replication-based strategies. Although in this work we focus on a simplified setting where the storage system stores a single content, the methodology developed can be applied to more general settings with multiple contents. The results also offer insightful guidance to the design of storage systems in data centers and content distribution networks. △ Less

Submitted 7 February, 2012; originally announced February 2012.

Regenerating Codes for Errors and Erasures in Distributed Storage

Authors: K. V. Rashmi, Nihar B. Shah, Kannan Ramchandran, P. Vijay Kumar

Abstract: Regenerating codes are a class of codes proposed for providing reliability of data and efficient repair of failed nodes in distributed storage systems. In this paper, we address the fundamental problem of handling errors and erasures during the data-reconstruction and node-repair operations. We provide explicit regenerating codes that are resilient to errors and erasures, and show that these codes… ▽ More Regenerating codes are a class of codes proposed for providing reliability of data and efficient repair of failed nodes in distributed storage systems. In this paper, we address the fundamental problem of handling errors and erasures during the data-reconstruction and node-repair operations. We provide explicit regenerating codes that are resilient to errors and erasures, and show that these codes are optimal with respect to storage and bandwidth requirements. As a special case, we also establish the capacity of a class of distributed storage systems in the presence of malicious adversaries. While our code constructions are based on previously constructed Product-Matrix codes, we also provide necessary and sufficient conditions for introducing resilience in any regenerating code. △ Less

Submitted 23 May, 2012; v1 submitted 6 February, 2012; originally announced February 2012.

Comments: ISIT 2012

A Compression Algorithm Using Mis-aligned Side-information

Authors: Nan Ma, Kannan Ramchandran, David Tse

Abstract: We study the problem of compressing a source sequence in the presence of side-information that is related to the source via insertions, deletions and substitutions. We propose a simple algorithm to compress the source sequence when the side-information is present at both the encoder and decoder. A key attribute of the algorithm is that it encodes the edits contained in runs of different extents se… ▽ More We study the problem of compressing a source sequence in the presence of side-information that is related to the source via insertions, deletions and substitutions. We propose a simple algorithm to compress the source sequence when the side-information is present at both the encoder and decoder. A key attribute of the algorithm is that it encodes the edits contained in runs of different extents separately. For small insertion and deletion probabilities, the compression rate of the algorithm is shown to be asymptotically optimal. △ Less

Submitted 3 February, 2012; originally announced February 2012.

Comments: 7 pages, 2 figures, 1 table. A shorter version is submitted to IEEE International Symposium on Information Theory (ISIT), 2012

Optimal Power Procurement and Demand Response with Quality-of-Usage Guarantees

Authors: Longbo Huang, Jean Walrand, Kannan Ramchandran

Abstract: In this paper, we propose a general operating scheme which allows the utility company to jointly perform power procurement and demand response so as to maximize the social welfare. Our model takes into consideration the effect of the renewable energy and the multi-stage feature of the power procurement process. It also enables the utility company to provide quality-of-usage (QoU) guarantee to the… ▽ More In this paper, we propose a general operating scheme which allows the utility company to jointly perform power procurement and demand response so as to maximize the social welfare. Our model takes into consideration the effect of the renewable energy and the multi-stage feature of the power procurement process. It also enables the utility company to provide quality-of-usage (QoU) guarantee to the power consumers, which ensures that the average power usage level meets the target value for each user. To maximize the social welfare, we develop a low-complexity algorithm called the \emph{welfare maximization algorithm} (WMA), which performs joint power procurement and dynamic pricing. WMA is constructed based on a two-timescale Lyapunov optimization technique. We prove that WMA achieves a close-to-optimal utility and ensures that the QoU requirement is met with bounded deficit. WMA can be implemented in a distributed manner and is robust with respect to system dynamics uncertainty. △ Less

Submitted 3 December, 2011; originally announced December 2011.

Optimal Deterministic Polynomial-Time Data Exchange for Omniscience

Authors: Nebojsa Milosavljevic, Sameer Pawar, Salim El Rouayheb, Michael Gastpar, Kannan Ramchandran

Abstract: We study the problem of constructing a deterministic polynomial time algorithm that achieves omniscience, in a rate-optimal manner, among a set of users that are interested in a common file but each has only partial knowledge about it as side-information. Assuming that the collective information among all the users is sufficient to allow the reconstruction of the entire file, the goal is to minimi… ▽ More We study the problem of constructing a deterministic polynomial time algorithm that achieves omniscience, in a rate-optimal manner, among a set of users that are interested in a common file but each has only partial knowledge about it as side-information. Assuming that the collective information among all the users is sufficient to allow the reconstruction of the entire file, the goal is to minimize the (possibly weighted) amount of bits that these users need to exchange over a noiseless public channel in order for all of them to learn the entire file. Using established connections to the multi-terminal secrecy problem, our algorithm also implies a polynomial-time method for constructing a maximum size secret shared key in the presence of an eavesdropper. We consider the following types of side-information settings: (i) side information in the form of uncoded fragments/packets of the file, where the users' side-information consists of subsets of the file; (ii) side information in the form of linearly correlated packets, where the users have access to linear combinations of the file packets; and (iii) the general setting where the the users' side-information has an arbitrary (i.i.d.) correlation structure. Building on results from combinatorial optimization, we provide a polynomial-time algorithm (in the number of users) that, first finds the optimal rate allocations among these users, then determines an explicit transmission scheme (i.e., a description of which user should transmit what information) for cases (i) and (ii). △ Less

Submitted 30 August, 2011; originally announced August 2011.

Achievable Rates for Channels with Deletions and Insertions

Authors: Ramji Venkataramanan, Sekhar Tatikonda, Kannan Ramchandran

Abstract: This paper considers a binary channel with deletions and insertions, where each input bit is transformed in one of the following ways: it is deleted with probability d, or an extra bit is added after it with probability i, or it is transmitted unmodified with probability 1-d-i. A computable lower bound on the capacity of this channel is derived. The transformation of the input sequence by the chan… ▽ More This paper considers a binary channel with deletions and insertions, where each input bit is transformed in one of the following ways: it is deleted with probability d, or an extra bit is added after it with probability i, or it is transmitted unmodified with probability 1-d-i. A computable lower bound on the capacity of this channel is derived. The transformation of the input sequence by the channel may be viewed in terms of runs as follows: some runs of the input sequence get shorter/longer, some runs get deleted, and some new runs are added. It is difficult for the decoder to synchronize the channel output sequence to the transmitted codeword mainly due to deleted runs and new inserted runs. The main idea is a mutual information decomposition in terms of the rate achieved by a sub-optimal decoder that determines the positions of the deleted and inserted runs in addition to decoding the transmitted codeword. The mutual information between the channel input and output sequences is expressed as the sum of the rate achieved by this decoder and the rate loss due to its sub-optimality. Obtaining computable lower bounds on each of these quantities yields a lower bound on the capacity. The bounds proposed in this paper provide the first characterization of achievable rates for channels with general insertions, and for channels with both deletions and insertions. For the special case of the deletion channel, the proposed bound improves on the previous best lower bound for deletion probabilities up to 0.3. △ Less

Submitted 19 July, 2013; v1 submitted 24 February, 2011; originally announced February 2011.

Comments: To appear in IEEE Transactions on Information Theory. For the deletion channel, the new capacity lower bound improves on the previous best bound for deletion probabilities up to 0.3

Journal ref: IEEE Transactions on Information Theory, vol. 59, no.11, pp. 6990-7013, Nov. 2013

Efficient File Synchronization: a Distributed Source Coding Approach

Authors: Nan Ma, Kannan Ramchandran, David Tse

Abstract: The problem of reconstructing a source sequence with the presence of decoder side-information that is mis-synchronized to the source due to deletions is studied in a distributed source coding framework. Motivated by practical applications, the deletion process is assumed to be bursty and is modeled by a Markov chain. The minimum rate needed to reconstruct the source sequence with high probability… ▽ More The problem of reconstructing a source sequence with the presence of decoder side-information that is mis-synchronized to the source due to deletions is studied in a distributed source coding framework. Motivated by practical applications, the deletion process is assumed to be bursty and is modeled by a Markov chain. The minimum rate needed to reconstruct the source sequence with high probability is characterized in terms of an information theoretic expression, which is interpreted as the amount of information of the deleted content and the locations of deletions, subtracting "nature's secret", that is, the uncertainty of the locations given the source and side-information. For small bursty deletion probability, the asymptotic expansion of the minimum rate is computed. △ Less

Submitted 21 July, 2011; v1 submitted 17 February, 2011; originally announced February 2011.

Comments: 9 pages, 2 figures. A shorter version will appear in IEEE International Symposium on Information Theory (ISIT), 2011

Hybrid Digital-Analog Codes for Source-Channel Broadcast of Gaussian Sources over Gaussian Channels

Authors: Vinod M. Prabhakaran, Rohit Puri, Kannan Ramchandran

Abstract: The problem of broadcasting a parallel Gaussian source over an additive white Gaussian noise broadcast channel under the mean-squared error distortion criterion is studied. A hybrid digital-analog coding strategy which combines source coding with side information, channel coding with side information, layered source coding, and superposition broadcast channel coding is presented. When specialized… ▽ More The problem of broadcasting a parallel Gaussian source over an additive white Gaussian noise broadcast channel under the mean-squared error distortion criterion is studied. A hybrid digital-analog coding strategy which combines source coding with side information, channel coding with side information, layered source coding, and superposition broadcast channel coding is presented. When specialized to the open problem of broadcasting a white Gaussian source over an additive white Gaussian noise broadcast channel with bandwidth mismatch which has been the subject of several previous investigations, this coding scheme strictly improves on the state-of-the-art. △ Less

Submitted 26 January, 2011; originally announced January 2011.

Comments: 24 pages, 9 figures

Distributed High Accuracy Peer-to-Peer Localization in Mobile Multipath Environments

Authors: Venkatesan Ekambaram, Kannan Ramchandran

Abstract: In this paper we consider the problem of high accuracy localization of mobile nodes in a multipath-rich environment where sub-meter accuracies are required. We employ a peer to peer framework where the vehicles/nodes can get pairwise multipath-degraded ranging estimates in local neighborhoods together with a fixed number of anchor nodes. The challenge is to overcome the multipath-barrier with redu… ▽ More In this paper we consider the problem of high accuracy localization of mobile nodes in a multipath-rich environment where sub-meter accuracies are required. We employ a peer to peer framework where the vehicles/nodes can get pairwise multipath-degraded ranging estimates in local neighborhoods together with a fixed number of anchor nodes. The challenge is to overcome the multipath-barrier with redundancy in order to provide the desired accuracies especially under severe multipath conditions when the fraction of received signals corrupted by multipath is dominating. We invoke a message passing analytical framework based on particle filtering and reveal its high accuracy localization promise through simulations. △ Less

Submitted 28 November, 2010; originally announced November 2010.

Comments: 5 pages, 5 figures, Accepted at IEEE Globecom 2010, Miami, FL

An Adaptive Multi-channel P2P Video-on-Demand System using Plug-and-Play Helpers

Authors: Hao Zhang, Minghua Chen, Abhay Parekh, Kannan Ramchandran

Abstract: We present a multi-channel P2P Video-on-Demand (VoD) system using "plug-and-play" helpers. Helpers are heterogenous "micro-servers" with limited storage, bandwidth and number of users they can serve simultaneously. Our proposed system has the following salient features: (1) it minimizes the server load; (2) it is distributed, and requires little or no maintenance overhead and which can easily adap… ▽ More We present a multi-channel P2P Video-on-Demand (VoD) system using "plug-and-play" helpers. Helpers are heterogenous "micro-servers" with limited storage, bandwidth and number of users they can serve simultaneously. Our proposed system has the following salient features: (1) it minimizes the server load; (2) it is distributed, and requires little or no maintenance overhead and which can easily adapt to system dynamics; and (3) it is adaptable to varying supply and demand patterns across multiple video channels irrespective of video popularity. Our proposed solution jointly optimizes over helper-user topology, video storage allocation and bandwidth allocation. The combinatorial nature of the problem and the system demand for distributed algorithms makes the problem uniquely challenging. By utilizing Lagrangian decomposition and Markov chain approximation based arguments, we address this challenge by designing two distributed algorithms running in tandem: a primal-dual storage and bandwidth allocation algorithm and a "soft-worst-neighbor-choking" topology-building algorithm. Our scheme provably converges to a near-optimal solution, and is easy to implement in practice. Simulation results validate that the proposed scheme achieves minimum sever load under highly heterogeneous combinations of supply and demand patterns, and is robust to system dynamics of user/helper churn, user/helper asynchrony, and random delays in the network. △ Less

Submitted 24 November, 2010; originally announced November 2010.