-
Almost Optimal Scaling of Reed-Muller Codes on BEC and BSC Channels
Authors:
Hamed Hassani,
Shrinivas Kudekar,
Or Ordentlich,
Yury Polyanskiy,
Rüdiger Urbanke
Abstract:
Consider a binary linear code of length $N$, minimum distance $d_{\text{min}}$, transmission over the binary erasure channel with parameter $0 < ε< 1$ or the binary symmetric channel with parameter $0 < ε< \frac12$, and block-MAP decoding. It was shown by Tillich and Zemor that in this case the error probability of the block-MAP decoder transitions "quickly" from $δ$ to $1-δ$ for any $δ>0$ if the…
▽ More
Consider a binary linear code of length $N$, minimum distance $d_{\text{min}}$, transmission over the binary erasure channel with parameter $0 < ε< 1$ or the binary symmetric channel with parameter $0 < ε< \frac12$, and block-MAP decoding. It was shown by Tillich and Zemor that in this case the error probability of the block-MAP decoder transitions "quickly" from $δ$ to $1-δ$ for any $δ>0$ if the minimum distance is large. In particular the width of the transition is of order $O(1/\sqrt{d_{\text{min}}})$. We strengthen this result by showing that under suitable conditions on the weight distribution of the code, the transition width can be as small as $Θ(1/N^{\frac12-κ})$, for any $κ>0$, even if the minimum distance of the code is not linear. This condition applies e.g., to Reed-Mueller codes. Since $Θ(1/N^{\frac12})$ is the smallest transition possible for any code, we speak of "almost" optimal scaling. We emphasize that the width of the transition says nothing about the location of the transition. Therefore this result has no bearing on whether a code is capacity-achieving or not. As a second contribution, we present a new estimate on the derivative of the EXIT function, the proof of which is based on the Blowing-Up Lemma.
△ Less
Submitted 29 January, 2018;
originally announced January 2018.
-
Comparing the Bit-MAP and Block-MAP Decoding Thresholds of Reed-Muller Codes on BMS Channels
Authors:
Shrinivas Kudekar,
Santhosh Kumar,
Marco Mondelli,
Henry D. Pfister,
Rüdiger Urbanke
Abstract:
The question whether RM codes are capacity-achieving is a long-standing open problem in coding theory that was recently answered in the affirmative for transmission over erasure channels [1], [2]. Remarkably, the proof does not rely on specific properties of RM codes, apart from their symmetry. Indeed, the main technical result consists in showing that any sequence of linear codes, with doubly-tra…
▽ More
The question whether RM codes are capacity-achieving is a long-standing open problem in coding theory that was recently answered in the affirmative for transmission over erasure channels [1], [2]. Remarkably, the proof does not rely on specific properties of RM codes, apart from their symmetry. Indeed, the main technical result consists in showing that any sequence of linear codes, with doubly-transitive permutation groups, achieves capacity on the memoryless erasure channel under bit-MAP decoding. Thus, a natural question is what happens under block-MAP decoding. In [1], [2], by exploiting further symmetries of the code, the bit-MAP threshold was shown to be sharp enough so that the block erasure probability also converges to 0. However, this technique relies heavily on the fact that the transmission is over an erasure channel.
We present an alternative approach to strengthen results regarding the bit-MAP threshold to block-MAP thresholds. This approach is based on a careful analysis of the weight distribution of RM codes. In particular, the flavor of the main result is the following: assume that the bit-MAP error probability decays as $N^{-δ}$, for some $δ>0$. Then, the block-MAP error probability also converges to 0. This technique applies to transmission over any binary memoryless symmetric channel. Thus, it can be thought of as a first step in extending the proof that RM codes are capacity-achieving to the general case.
△ Less
Submitted 9 July, 2016; v1 submitted 22 January, 2016;
originally announced January 2016.
-
Joint Crosstalk-Avoidance and Error-Correction Coding for Parallel Data Buses
Authors:
Urs Niesen,
Shrinivas Kudekar
Abstract:
Decreasing transistor sizes and lower voltage swings cause two distinct problems for communication in integrated circuits. First, decreasing inter-wire spacing increases interline capacitive coupling, which adversely affects transmission energy and delay. Second, lower voltage swings render the transmission susceptible to various noise sources. Coding can be used to address both these problems. So…
▽ More
Decreasing transistor sizes and lower voltage swings cause two distinct problems for communication in integrated circuits. First, decreasing inter-wire spacing increases interline capacitive coupling, which adversely affects transmission energy and delay. Second, lower voltage swings render the transmission susceptible to various noise sources. Coding can be used to address both these problems. So-called crosstalk-avoidance codes mitigate capacitive coupling, and traditional error-correction codes introduce resilience against channel errors.
Unfortunately, crosstalk-avoidance and error-correction codes cannot be combined in a straightforward manner. On the one hand, crosstalk-avoidance encoding followed by error-correction encoding destroys the crosstalk-avoidance property. On the other hand, error-correction encoding followed by crosstalk-avoidance encoding causes the crosstalk-avoidance decoder to fail in the presence of errors. Existing approaches circumvent this difficulty by using additional bus wires to protect the parities generated from the output of the error-correction encoder, and are therefore inefficient.
In this work we propose a novel joint crosstalk-avoidance and error-correction coding and decoding scheme that provides higher bus transmission rates compared to existing approaches. Our joint approach carefully embeds the parities such that the crosstalk-avoidance property is preserved. We analyze the rate and minimum distance of the proposed scheme. We also provide a density evolution analysis and predict iterative decoding thresholds for reliable communication under random bus erasures. This density evolution analysis is nonstandard, since the crosstalk-avoidance constraints are inherently nonlinear.
△ Less
Submitted 19 January, 2016;
originally announced January 2016.
-
Reed-Muller Codes Achieve Capacity on Erasure Channels
Authors:
Shrinivas Kudekar,
Santhosh Kumar,
Marco Mondelli,
Henry D. Pfister,
Eren Şaşoğlu,
Rüdiger Urbanke
Abstract:
We introduce a new approach to proving that a sequence of deterministic linear codes achieves capacity on an erasure channel under maximum a posteriori decoding. Rather than relying on the precise structure of the codes our method exploits code symmetry. In particular, the technique applies to any sequence of linear codes where the blocklengths are strictly increasing, the code rates converge, and…
▽ More
We introduce a new approach to proving that a sequence of deterministic linear codes achieves capacity on an erasure channel under maximum a posteriori decoding. Rather than relying on the precise structure of the codes our method exploits code symmetry. In particular, the technique applies to any sequence of linear codes where the blocklengths are strictly increasing, the code rates converge, and the permutation group of each code is doubly transitive. In other words, we show that symmetry alone implies near-optimal performance.
An important consequence of this result is that a sequence of Reed-Muller codes with increasing blocklength and converging rate achieves capacity. This possibility has been suggested previously in the literature but it has only been proven for cases where the limiting code rate is 0 or 1. Moreover, these results extend naturally to all affine-invariant codes and, thus, to extended primitive narrow-sense BCH codes. This also resolves, in the affirmative, the existence question for capacity-achieving sequences of binary cyclic codes. The primary tools used in the proof are the sharp threshold property for symmetric monotone boolean functions and the area theorem for extrinsic information transfer functions.
△ Less
Submitted 18 January, 2016;
originally announced January 2016.
-
Reed-Muller Codes Achieve Capacity on the Binary Erasure Channel under MAP Decoding
Authors:
Shrinivas Kudekar,
Marco Mondelli,
Eren Şaşoğlu,
Rüdiger Urbanke
Abstract:
We show that Reed-Muller codes achieve capacity under maximum a posteriori bit decoding for transmission over the binary erasure channel for all rates $0 < R < 1$. The proof is generic and applies to other codes with sufficient amount of symmetry as well. The main idea is to combine the following observations: (i) monotone functions experience a sharp threshold behavior, (ii) the extrinsic informa…
▽ More
We show that Reed-Muller codes achieve capacity under maximum a posteriori bit decoding for transmission over the binary erasure channel for all rates $0 < R < 1$. The proof is generic and applies to other codes with sufficient amount of symmetry as well. The main idea is to combine the following observations: (i) monotone functions experience a sharp threshold behavior, (ii) the extrinsic information transfer (EXIT) functions are monotone, (iii) Reed--Muller codes are 2-transitive and thus the EXIT functions associated with their codeword bits are all equal, and (iv) therefore the Area Theorem for the average EXIT functions implies that RM codes' threshold is at channel capacity.
△ Less
Submitted 21 May, 2015;
originally announced May 2015.
-
Analysis of Saturated Belief Propagation Decoding of Low-Density Parity-Check Codes
Authors:
Shrinivas Kudekar,
Tom Richardson,
Aravind Iyengar
Abstract:
We consider the effect of log-likelihood ratio saturation on belief propagation decoder low-density parity-check codes. Saturation is commonly done in practice and is known to have a significant effect on error floor performance. Our focus is on threshold analysis and stability of density evolution.
We analyze the decoder for standard low-density parity-check code ensembles and show that belief…
▽ More
We consider the effect of log-likelihood ratio saturation on belief propagation decoder low-density parity-check codes. Saturation is commonly done in practice and is known to have a significant effect on error floor performance. Our focus is on threshold analysis and stability of density evolution.
We analyze the decoder for standard low-density parity-check code ensembles and show that belief propagation decoding generally degrades gracefully with saturation. Stability of density evolution is, on the other hand, rather strongly effected by saturation and the asymptotic qualitative effect of saturation is similar to reduction by one of variable node degree.
We also show under what conditions the block threshold for the saturated belief propagation corresponds with the bit threshold.
△ Less
Submitted 27 December, 2014;
originally announced December 2014.
-
The Effect of Saturation on Belief Propagation Decoding of LDPC Codes
Authors:
Shrinivas Kudekar,
Tom Richardson,
Aravind Iyengar
Abstract:
We consider the effect of LLR saturation on belief propagation decoding of low-density parity-check codes. Saturation occurs universally in practice and is known to have a significant effect on error floor performance. Our focus is on threshold analysis and stability of density evolution.
We analyze the decoder for certain low-density parity-check code ensembles and show that belief propagation…
▽ More
We consider the effect of LLR saturation on belief propagation decoding of low-density parity-check codes. Saturation occurs universally in practice and is known to have a significant effect on error floor performance. Our focus is on threshold analysis and stability of density evolution.
We analyze the decoder for certain low-density parity-check code ensembles and show that belief propagation decoding generally degrades gracefully with saturation. Stability of density evolution is, on the other hand, rather strongly affected by saturation and the asymptotic qualitative effect of saturation is similar to reduction of variable node degree by one.
△ Less
Submitted 14 March, 2014;
originally announced March 2014.
-
Wave-Like Solutions of General One-Dimensional Spatially Coupled Systems
Authors:
Shrinivas Kudekar,
Tom Richardson,
Ruediger Urbanke
Abstract:
We establish the existence of wave-like solutions to spatially coupled graphical models which, in the large size limit, can be characterized by a one-dimensional real-valued state. This is extended to a proof of the threshold saturation phenomenon for all such models, which includes spatially coupled irregular LDPC codes over the BEC, but also addresses hard-decision decoding for transmission over…
▽ More
We establish the existence of wave-like solutions to spatially coupled graphical models which, in the large size limit, can be characterized by a one-dimensional real-valued state. This is extended to a proof of the threshold saturation phenomenon for all such models, which includes spatially coupled irregular LDPC codes over the BEC, but also addresses hard-decision decoding for transmission over general channels, the CDMA multiple-access problem, compressed sensing, and some statistical physics models.
For traditional uncoupled iterative coding systems with two components and transmission over the BEC, the asymptotic convergence behavior is completely characterized by the EXIT curves of the components. More precisely, the system converges to the desired fixed point, which is the one corresponding to perfect decoding, if and only if the two EXIT functions describing the components do not cross. For spatially coupled systems whose state is one-dimensional a closely related graphical criterion applies. Now the curves are allowed to cross, but not by too much. More precisely, we show that the threshold saturation phenomenon is related to the positivity of the (signed) area enclosed by two EXIT-like functions associated to the component systems, a very intuitive and easy-to-use graphical characterization.
In the spirit of EXIT functions and Gaussian approximations, we also show how to apply the technique to higher dimensional and even infinite-dimensional cases. In these scenarios the method is no longer rigorous, but it typically gives accurate predictions. To demonstrate this application, we discuss transmission over general channels using both the belief-propagation as well as the min-sum decoder.
△ Less
Submitted 27 October, 2014; v1 submitted 26 August, 2012;
originally announced August 2012.
-
Spatially Coupled Ensembles Universally Achieve Capacity under Belief Propagation
Authors:
Shrinivas Kudekar,
Tom Richardson,
Ruediger Urbanke
Abstract:
We investigate spatially coupled code ensembles. For transmission over the binary erasure channel, it was recently shown that spatial coupling increases the belief propagation threshold of the ensemble to essentially the maximum a-priori threshold of the underlying component ensemble. This explains why convolutional LDPC ensembles, originally introduced by Felstrom and Zigangirov, perform so well…
▽ More
We investigate spatially coupled code ensembles. For transmission over the binary erasure channel, it was recently shown that spatial coupling increases the belief propagation threshold of the ensemble to essentially the maximum a-priori threshold of the underlying component ensemble. This explains why convolutional LDPC ensembles, originally introduced by Felstrom and Zigangirov, perform so well over this channel. We show that the equivalent result holds true for transmission over general binary-input memoryless output-symmetric channels. More precisely, given a desired error probability and a gap to capacity, we can construct a spatially coupled ensemble which fulfills these constraints universally on this class of channels under belief propagation decoding. In fact, most codes in that ensemble have that property. The quantifier universal refers to the single ensemble/code which is good for all channels but we assume that the channel is known at the receiver. The key technical result is a proof that under belief propagation decoding spatially coupled ensembles achieve essentially the area threshold of the underlying uncoupled ensemble. We conclude by discussing some interesting open problems.
△ Less
Submitted 14 January, 2012;
originally announced January 2012.
-
Improved Linear Programming Decoding using Frustrated Cycles
Authors:
Shrinivas Kudekar,
Jason K. Johnson,
Misha Chertkov
Abstract:
We consider transmission over a binary-input additive white Gaussian noise channel using low-density parity-check codes. One of the most popular techniques for decoding low-density parity-check codes is the linear programming decoder. In general, the linear programming decoder is suboptimal. I.e., the word error rate is higher than the optimal, maximum a posteriori decoder.
In this paper we pres…
▽ More
We consider transmission over a binary-input additive white Gaussian noise channel using low-density parity-check codes. One of the most popular techniques for decoding low-density parity-check codes is the linear programming decoder. In general, the linear programming decoder is suboptimal. I.e., the word error rate is higher than the optimal, maximum a posteriori decoder.
In this paper we present a systematic approach to enhance the linear program decoder. More precisely, in the cases where the linear program outputs a fractional solution, we give a simple algorithm to identify frustrated cycles which cause the output of the linear program to be fractional. Then adding these cycles, adaptively to the basic linear program, we show improved word error rate performance.
△ Less
Submitted 23 May, 2011;
originally announced May 2011.
-
Linear Programming based Detectors for Two-Dimensional Intersymbol Interference Channels
Authors:
Shrinivas Kudekar,
Jason K. Johnson,
Michael Chertkov
Abstract:
We present and study linear programming based detectors for two-dimensional intersymbol interference channels. Interesting instances of two-dimensional intersymbol interference channels are magnetic storage, optical storage and Wyner's cellular network model.
We show that the optimal maximum a posteriori detection in such channels lends itself to a natural linear programming based sub-optimal de…
▽ More
We present and study linear programming based detectors for two-dimensional intersymbol interference channels. Interesting instances of two-dimensional intersymbol interference channels are magnetic storage, optical storage and Wyner's cellular network model.
We show that the optimal maximum a posteriori detection in such channels lends itself to a natural linear programming based sub-optimal detector. We call this the Pairwise linear program detector. Our experiments show that the Pairwise linear program detector performs poorly. We then propose two methods to strengthen our detector. These detectors are based on systematically enhancing the Pairwise linear program. The first one, the Block linear program detector adds higher order potential functions in an {\em exhaustive} manner, as constraints, to the Pairwise linear program detector. We show by experiments that the Block linear program detector has performance close to the optimal detector. We then develop another detector by
{\em adaptively} adding frustrated cycles to the Pairwise linear program detector. Empirically, this detector also has performance close to the optimal one and turns out to be less complex then the Block linear program detector.
△ Less
Submitted 25 February, 2011;
originally announced February 2011.
-
Spatially Coupled Codes over the Multiple Access Channel
Authors:
Shrinivas Kudekar,
Kenta Kasai
Abstract:
We consider spatially coupled code ensembles over a multiple access channel. Convolutional LDPC ensembles are one instance of spatially coupled codes. It was shown recently that, for transmission over the binary erasure channel, this coupling of individual code ensembles has the effect of increasing the belief propagation threshold of the coupled ensembles to the maximum a-posteriori threshold of…
▽ More
We consider spatially coupled code ensembles over a multiple access channel. Convolutional LDPC ensembles are one instance of spatially coupled codes. It was shown recently that, for transmission over the binary erasure channel, this coupling of individual code ensembles has the effect of increasing the belief propagation threshold of the coupled ensembles to the maximum a-posteriori threshold of the underlying ensemble. In this sense, spatially coupled codes were shown to be capacity achieving. It was observed, empirically, that these codes are universal in the sense that they achieve performance close to the Shannon threshold for any general binary-input memoryless symmetric channels.
In this work we provide further evidence of the threshold saturation phenomena when transmitting over a class of multiple access channel. We show, by density evolution analysis and EXIT curves, that the belief propagation threshold of the coupled ensembles is very close to the ultimate Shannon limit.
△ Less
Submitted 14 February, 2011;
originally announced February 2011.
-
Threshold Saturation on Channels with Memory via Spatial Coupling
Authors:
Shrinivas Kudekar,
Kenta Kasai
Abstract:
We consider spatially coupled code ensembles. A particular instance are convolutional LDPC ensembles. It was recently shown that, for transmission over the memoryless binary erasure channel, this coupling increases the belief propagation threshold of the ensemble to the maximum a-posteriori threshold of the underlying component ensemble. This paved the way for a new class of capacity achieving low…
▽ More
We consider spatially coupled code ensembles. A particular instance are convolutional LDPC ensembles. It was recently shown that, for transmission over the memoryless binary erasure channel, this coupling increases the belief propagation threshold of the ensemble to the maximum a-posteriori threshold of the underlying component ensemble. This paved the way for a new class of capacity achieving low-density parity check codes. It was also shown empirically that the same threshold saturation occurs when we consider transmission over general binary input memoryless channels.
In this work, we report on empirical evidence which suggests that the same phenomenon also occurs when transmission takes place over a class of channels with memory. This is confirmed both by simulations as well as by computing EXIT curves.
△ Less
Submitted 2 February, 2011;
originally announced February 2011.
-
The Effect of Spatial Coupling on Compressive Sensing
Authors:
Shrinivas Kudekar,
Henry D. Pfister
Abstract:
Recently, it was observed that spatially-coupled LDPC code ensembles approach the Shannon capacity for a class of binary-input memoryless symmetric (BMS) channels. The fundamental reason for this was attributed to a "threshold saturation" phenomena derived by Kudekar, Richardson and Urbanke. In particular, it was shown that the belief propagation (BP) threshold of the spatially coupled codes is eq…
▽ More
Recently, it was observed that spatially-coupled LDPC code ensembles approach the Shannon capacity for a class of binary-input memoryless symmetric (BMS) channels. The fundamental reason for this was attributed to a "threshold saturation" phenomena derived by Kudekar, Richardson and Urbanke. In particular, it was shown that the belief propagation (BP) threshold of the spatially coupled codes is equal to the maximum a posteriori (MAP) decoding threshold of the underlying constituent codes. In this sense, the BP threshold is saturated to its maximum value. Moreover, it has been empirically observed that the same phenomena also occurs when transmitting over more general classes of BMS channels. In this paper, we show that the effect of spatial coupling is not restricted to the realm of channel coding. The effect of coupling also manifests itself in compressed sensing. Specifically, we show that spatially-coupled measurement matrices have an improved sparsity to sampling threshold for reconstruction algorithms based on verification decoding. For BP-based reconstruction algorithms, this phenomenon is also tested empirically via simulation. At the block lengths accessible via simulation, the effect is quite small and it seems that spatial coupling is not providing the gains one might expect. Based on the threshold analysis, however, we believe this warrants further study.
△ Less
Submitted 28 October, 2010;
originally announced October 2010.
-
Threshold Saturation on BMS Channels via Spatial Coupling
Authors:
Shrinivas Kudekar,
Cyril Measson,
Tom Richardson,
Ruediger Urbanke
Abstract:
We consider spatially coupled code ensembles. A particular instance are convolutional LDPC ensembles. It was recently shown that, for transmission over the binary erasure channel, this coupling increases the belief propagation threshold of the ensemble to the maximum a-priori threshold of the underlying component ensemble. We report on empirical evidence which suggest that the same phenomenon al…
▽ More
We consider spatially coupled code ensembles. A particular instance are convolutional LDPC ensembles. It was recently shown that, for transmission over the binary erasure channel, this coupling increases the belief propagation threshold of the ensemble to the maximum a-priori threshold of the underlying component ensemble. We report on empirical evidence which suggest that the same phenomenon also occurs when transmission takes place over a general binary memoryless symmetric channel. This is confirmed both by simulations as well as by computing EBP GEXIT curves and by comparing the empirical BP thresholds of coupled ensembles to the empirically determined MAP thresholds of the underlying regular ensembles. We further consider ways of reducing the rate-loss incurred by such constructions.
△ Less
Submitted 21 April, 2010;
originally announced April 2010.
-
Threshold Saturation via Spatial Coupling: Why Convolutional LDPC Ensembles Perform so well over the BEC
Authors:
Shrinivas Kudekar,
Tom Richardson,
Ruediger Urbanke
Abstract:
Convolutional LDPC ensembles, introduced by Felstrom and Zigangirov, have excellent thresholds and these thresholds are rapidly increasing as a function of the average degree. Several variations on the basic theme have been proposed to date, all of which share the good performance characteristics of convolutional LDPC ensembles. We describe the fundamental mechanism which explains why "convolution…
▽ More
Convolutional LDPC ensembles, introduced by Felstrom and Zigangirov, have excellent thresholds and these thresholds are rapidly increasing as a function of the average degree. Several variations on the basic theme have been proposed to date, all of which share the good performance characteristics of convolutional LDPC ensembles. We describe the fundamental mechanism which explains why "convolutional-like" or "spatially coupled" codes perform so well. In essence, the spatial coupling of the individual code structure has the effect of increasing the belief-propagation (BP) threshold of the new ensemble to its maximum possible value, namely the maximum-a-posteriori (MAP) threshold of the underlying ensemble. For this reason we call this phenomenon "threshold saturation." This gives an entirely new way of approaching capacity. One significant advantage of such a construction is that one can create capacity-approaching ensembles with an error correcting radius which is increasing in the blocklength. Our proof makes use of the area theorem of the BP-EXIT curve and the connection between the MAP and BP threshold recently pointed out by Measson, Montanari, Richardson, and Urbanke. Although we prove the connection between the MAP and the BP threshold only for a very specific ensemble and only for the binary erasure channel, empirically a threshold saturation phenomenon occurs for a wide class of ensembles and channels. More generally, we conjecture that for a large range of graphical systems a similar saturation of the "dynamical" threshold occurs once individual components are coupled sufficiently strongly. This might give rise to improved algorithms as well as to new techniques for analysis.
△ Less
Submitted 26 October, 2010; v1 submitted 12 January, 2010;
originally announced January 2010.
-
Decay of Correlations for Sparse Graph Error Correcting Codes
Authors:
Shrinivas Kudekar,
Nicolas Macris
Abstract:
The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code bits, and its relationship to the sub-optimal belief propagation decoder, are investigat…
▽ More
The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code bits, and its relationship to the sub-optimal belief propagation decoder, are investigated. We consider the correlation (or covariance) between two codebits, averaged over the noise realizations, as a function of the graph distance, for the optimal decoder. Our main result is that this correlation decays exponentially fast for fixed general low-density generator-matrix codes and high enough noise parameter, and also for fixed general low-density parity-check codes and low enough noise parameter. This has many consequences. Appropriate performance curves - called GEXIT functions - of the belief propagation and optimal decoders match in high/low noise regimes. This means that in high/low noise regimes the performance curves of the optimal decoder can be computed by density evolution. Another interpretation is that the replica predictions of spin-glass theory are exact. Our methods are rather general and use cluster expansions first developed in the context of mathematical statistical mechanics.
△ Less
Submitted 10 March, 2009;
originally announced March 2009.
-
Decay of Correlations in Low Density Parity Check Codes: Low Noise Regime
Authors:
Shrinivas Kudekar,
Nicolas Macris
Abstract:
Consider transmission over a binary additive white gaussian noise channel using a fixed low-density parity check code. We consider the posterior measure over the code bits and the corresponding correlation between two codebits, averaged over the noise realizations. We show that for low enough noise variance this average correlation decays exponentially fast with the graph distance between the co…
▽ More
Consider transmission over a binary additive white gaussian noise channel using a fixed low-density parity check code. We consider the posterior measure over the code bits and the corresponding correlation between two codebits, averaged over the noise realizations. We show that for low enough noise variance this average correlation decays exponentially fast with the graph distance between the code bits. One consequence of this result is that for low enough noise variance the GEXIT functions (further averaged over a standard code ensemble) of the belief propagation and optimal decoders are the same.
△ Less
Submitted 23 January, 2009;
originally announced January 2009.
-
Algebraic gossip on Arbitrary Networks
Authors:
Dinkar Vasudevan,
Shrinivas Kudekar
Abstract:
Consider a network of nodes where each node has a message to communicate to all other nodes. For this communication problem, we analyze a gossip based protocol where coded messages are exchanged. This problem was studied by Aoyama and Shah where a bound to the dissemination time based on the spectral properties of the underlying communication graph is provided. Our contribution is a uniform boun…
▽ More
Consider a network of nodes where each node has a message to communicate to all other nodes. For this communication problem, we analyze a gossip based protocol where coded messages are exchanged. This problem was studied by Aoyama and Shah where a bound to the dissemination time based on the spectral properties of the underlying communication graph is provided. Our contribution is a uniform bound that holds for arbitrary networks.
△ Less
Submitted 15 December, 2011; v1 submitted 11 January, 2009;
originally announced January 2009.
-
Sharp Bounds for Optimal Decoding of Low Density Parity Check Codes
Authors:
Shrinivas Kudekar,
Nicolas Macris
Abstract:
Consider communication over a binary-input memoryless output-symmetric channel with low density parity check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic formula for the average input-output conditional entropy per bit in the infinite block length limit. Montanari proved a lower bound for this entropy, in the case…
▽ More
Consider communication over a binary-input memoryless output-symmetric channel with low density parity check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic formula for the average input-output conditional entropy per bit in the infinite block length limit. Montanari proved a lower bound for this entropy, in the case of LDPC ensembles with convex check degree polynomial, which matches the replica formula. Here we extend this lower bound to any irregular LDPC ensemble. The new feature of our work is an analysis of the second derivative of the conditional input-output entropy with respect to noise. A close relation arises between this second derivative and correlation or mutual information of codebits. This allows us to extend the realm of the interpolation method, in particular we show how channel symmetry allows to control the fluctuations of the overlap parameters.
△ Less
Submitted 19 July, 2008;
originally announced July 2008.
-
Lower Bounds on the Rate-Distortion Function of Individual LDGM Codes
Authors:
Shrinivas Kudekar,
Ruediger Urbanke
Abstract:
We consider lossy compression of a binary symmetric source by means of a low-density generator-matrix code. We derive two lower bounds on the rate distortion function which are valid for any low-density generator-matrix code with a given node degree distribution L(x) on the set of generators and for any encoding algorithm. These bounds show that, due to the sparseness of the code, the performanc…
▽ More
We consider lossy compression of a binary symmetric source by means of a low-density generator-matrix code. We derive two lower bounds on the rate distortion function which are valid for any low-density generator-matrix code with a given node degree distribution L(x) on the set of generators and for any encoding algorithm. These bounds show that, due to the sparseness of the code, the performance is strictly bounded away from the Shannon rate-distortion function. In this sense, our bounds represent a natural generalization of Gallager's bound on the maximum rate at which low-density parity-check codes can be used for reliable transmission. Our bounds are similar in spirit to the technique recently developed by Dimakis, Wainwright, and Ramchandran, but they apply to individual codes.
△ Less
Submitted 10 April, 2008;
originally announced April 2008.
