-
Degree-$M$ Bethe and Sinkhorn Permanent Based Bounds on the Permanent of a Non-negative Matrix
Authors:
Yuwen Huang,
Navin Kashyap,
Pascal O. Vontobel
Abstract:
The permanent of a non-negative square matrix can be well approximated by finding the minimum of the Bethe free energy functions associated with some suitably defined factor graph; the resulting approximation to the permanent is called the Bethe permanent. Vontobel gave a combinatorial characterization of the Bethe permanent via degree-$M$ Bethe permanents, which are based on degree-$M$ covers of…
▽ More
The permanent of a non-negative square matrix can be well approximated by finding the minimum of the Bethe free energy functions associated with some suitably defined factor graph; the resulting approximation to the permanent is called the Bethe permanent. Vontobel gave a combinatorial characterization of the Bethe permanent via degree-$M$ Bethe permanents, which are based on degree-$M$ covers of the underlying factor graph. In this paper, we prove a degree-$M$-Bethe-permanent-based lower bound on the permanent of a non-negative matrix, which solves a conjecture proposed by Vontobel in [IEEE Trans. Inf. Theory, Mar. 2013]. We also prove a degree-$M$-Bethe-permanent-based upper bound on the permanent of a non-negative matrix. In the limit $M \to \infty$, these lower and upper bounds yield known Bethe-permanent-based lower and upper bounds on the permanent of a non-negative matrix. Moreover, we prove similar results for an approximation to the permanent known as the (scaled) Sinkhorn permanent.
△ Less
Submitted 28 May, 2024; v1 submitted 4 June, 2023;
originally announced June 2023.
-
Double-cover-based analysis of the Bethe permanent of non-negative matrices
Authors:
Kit Shing Ng,
Pascal O. Vontobel
Abstract:
The permanent of a non-negative matrix appears naturally in many information processing scenarios. Because of the intractability of the permanent beyond small matrices, various approximation techniques have been developed in the past. In this paper, we study the Bethe approximation of the permanent and add to the body of literature showing that this approximation is very well behaved in many respe…
▽ More
The permanent of a non-negative matrix appears naturally in many information processing scenarios. Because of the intractability of the permanent beyond small matrices, various approximation techniques have been developed in the past. In this paper, we study the Bethe approximation of the permanent and add to the body of literature showing that this approximation is very well behaved in many respects. Our main technical tool are topological double covers of the normal factor graph whose partition function equals the permanent of interest, along with a transformation of these double covers.
△ Less
Submitted 30 May, 2022;
originally announced May 2022.
-
Constrained Secrecy Capacity of Finite-Input Intersymbol Interference Wiretap Channels
Authors:
Aria Nouri,
Reza Asvadi,
Jun Chen,
Pascal O. Vontobel
Abstract:
We consider reliable and secure communication over intersymbol interference wiretap channels (ISI-WTCs). In particular, we first derive an achievable secure rate for ISI-WTCs without imposing any constraints on the input distribution. Afterwards, we focus on the setup where the input distribution of the ISI-WTC is constrained to be a time-invariant finite-order Markov chain. Optimizing the paramet…
▽ More
We consider reliable and secure communication over intersymbol interference wiretap channels (ISI-WTCs). In particular, we first derive an achievable secure rate for ISI-WTCs without imposing any constraints on the input distribution. Afterwards, we focus on the setup where the input distribution of the ISI-WTC is constrained to be a time-invariant finite-order Markov chain. Optimizing the parameters of this Markov chain toward maximizing the achievable secure rates is a computationally intractable problem in general, and so, toward finding a local maximum, we propose an iterative algorithm that at every iteration replaces the secure rate function with a suitable~surrogate function whose maximum can be found efficiently. Although the secure rates achieved in the unconstrained setup are potentially larger than the secure rates achieved in the constrained setup, the latter setup has the advantage of leading to efficient algorithms for estimating and optimizing the achievable secure rates, and also has the benefit of being the basis of efficient coding schemes.
△ Less
Submitted 12 April, 2023; v1 submitted 9 February, 2021;
originally announced February 2021.
-
Improved Lower Bounds on the Size of Balls over Permutations with the Infinity Metric
Authors:
Moshe Schwartz,
Pascal O. Vontobel
Abstract:
We study the size (or volume) of balls in the metric space of permutations, $S_n$, under the infinity metric. We focus on the regime of balls with radius $r = ρ\cdot (n\!-\!1)$, $ρ\in [0,1]$, i.e., a radius that is a constant fraction of the maximum possible distance. We provide new lower bounds on the size of such balls. These new lower bounds reduce the asymptotic gap to the known upper bounds t…
▽ More
We study the size (or volume) of balls in the metric space of permutations, $S_n$, under the infinity metric. We focus on the regime of balls with radius $r = ρ\cdot (n\!-\!1)$, $ρ\in [0,1]$, i.e., a radius that is a constant fraction of the maximum possible distance. We provide new lower bounds on the size of such balls. These new lower bounds reduce the asymptotic gap to the known upper bounds to at most $0.029$ bits per symbol. Additionally, they imply an improved ball-packing bound for error-correcting codes, and an improved upper bound on the size of optimal covering codes.
△ Less
Submitted 20 April, 2017; v1 submitted 16 September, 2016;
originally announced September 2016.
-
Factor Graphs for Quantum Probabilities
Authors:
Hans-Andrea Loeliger,
Pascal O. Vontobel
Abstract:
A factor-graph representation of quantum-mechanical probabilities (involving any number of measurements) is proposed. Unlike standard statistical models, the proposed representation uses auxiliary variables (state variables) that are not random variables. All joint probability distributions are marginals of some complex-valued function $q$, and it is demonstrated how the basic concepts of quantum…
▽ More
A factor-graph representation of quantum-mechanical probabilities (involving any number of measurements) is proposed. Unlike standard statistical models, the proposed representation uses auxiliary variables (state variables) that are not random variables. All joint probability distributions are marginals of some complex-valued function $q$, and it is demonstrated how the basic concepts of quantum mechanics relate to factorizations and marginals of $q$.
△ Less
Submitted 12 June, 2017; v1 submitted 4 August, 2015;
originally announced August 2015.
-
Coding for Combined Block-Symbol Error Correction
Authors:
Ron M. Roth,
Pascal O. Vontobel
Abstract:
We design low-complexity error correction coding schemes for channels that introduce different types of errors and erasures: on the one hand, the proposed schemes can successfully deal with symbol errors and erasures, and, on the other hand, they can also successfully handle phased burst errors and erasures.
We design low-complexity error correction coding schemes for channels that introduce different types of errors and erasures: on the one hand, the proposed schemes can successfully deal with symbol errors and erasures, and, on the other hand, they can also successfully handle phased burst errors and erasures.
△ Less
Submitted 27 February, 2014; v1 submitted 7 February, 2013;
originally announced February 2013.
-
The Bethe Permanent of a Non-Negative Matrix
Authors:
Pascal O. Vontobel
Abstract:
It has recently been observed that the permanent of a non-negative square matrix, i.e., of a square matrix containing only non-negative real entries, can very well be approximated by solving a certain Bethe free energy function minimization problem with the help of the sum-product algorithm. We call the resulting approximation of the permanent the Bethe permanent. In this paper we give reasons why…
▽ More
It has recently been observed that the permanent of a non-negative square matrix, i.e., of a square matrix containing only non-negative real entries, can very well be approximated by solving a certain Bethe free energy function minimization problem with the help of the sum-product algorithm. We call the resulting approximation of the permanent the Bethe permanent. In this paper we give reasons why this approach to approximating the permanent works well. Namely, we show that the Bethe free energy function is convex and that the sum-product algorithm finds its minimum efficiently. We then discuss the fact that the permanent is lower bounded by the Bethe permanent, and we comment on potential upper bounds on the permanent based on the Bethe permanent. We also present a combinatorial characterization of the Bethe permanent in terms of permanents of so-called lifted versions of the matrix under consideration. Moreover, we comment on possibilities to modify the Bethe permanent so that it approximates the permanent even better, and we conclude the paper with some observations and conjectures about permanent-based pseudo-codewords and permanent-based kernels.
△ Less
Submitted 20 October, 2012; v1 submitted 21 July, 2011;
originally announced July 2011.
-
LDPC Codes for Compressed Sensing
Authors:
Alexandros G. Dimakis,
Roxana Smarandache,
Pascal O. Vontobel
Abstract:
We present a mathematical connection between channel coding and compressed sensing. In particular, we link, on the one hand, \emph{channel coding linear programming decoding (CC-LPD)}, which is a well-known relaxation o maximum-likelihood channel decoding for binary linear codes, and, on the other hand, \emph{compressed sensing linear programming decoding (CS-LPD)}, also known as basis pursuit, w…
▽ More
We present a mathematical connection between channel coding and compressed sensing. In particular, we link, on the one hand, \emph{channel coding linear programming decoding (CC-LPD)}, which is a well-known relaxation o maximum-likelihood channel decoding for binary linear codes, and, on the other hand, \emph{compressed sensing linear programming decoding (CS-LPD)}, also known as basis pursuit, which is a widely used linear programming relaxation for the problem of finding the sparsest solution of an under-determined system of linear equations. More specifically, we establis a tight connection between CS-LPD based on a zero-one measurement matrix over the reals and CC-LPD of the binary linear channel code that is obtained by viewing this measurement matrix as a binary parity-check matrix. This connection allows the translation of performance guarantees from one setup to the other. The main message of this paper is that parity-check matrices of "good" channel codes can be used as provably "good" measurement matrices under basis pursuit. In particular, we provide the first deterministic construction of compressed sensing measurement matrices with an order-optimal number of rows using high-girth low-density parity-check (LDPC) codes constructed by Gallager.
△ Less
Submitted 11 December, 2011; v1 submitted 2 December, 2010;
originally announced December 2010.
-
Counting in Graph Covers: A Combinatorial Characterization of the Bethe Entropy Function
Authors:
Pascal O. Vontobel
Abstract:
We present a combinatorial characterization of the Bethe entropy function of a factor graph, such a characterization being in contrast to the original, analytical, definition of this function. We achieve this combinatorial characterization by counting valid configurations in finite graph covers of the factor graph. Analogously, we give a combinatorial characterization of the Bethe partition functi…
▽ More
We present a combinatorial characterization of the Bethe entropy function of a factor graph, such a characterization being in contrast to the original, analytical, definition of this function. We achieve this combinatorial characterization by counting valid configurations in finite graph covers of the factor graph. Analogously, we give a combinatorial characterization of the Bethe partition function, whose original definition was also of an analytical nature. As we point out, our approach has similarities to the replica method, but also stark differences. The above findings are a natural backdrop for introducing a decoder for graph-based codes that we will call symbolwise graph-cover decoding, a decoder that extends our earlier work on blockwise graph-cover decoding. Both graph-cover decoders are theoretical tools that help towards a better understanding of message-passing iterative decoding, namely blockwise graph-cover decoding links max-product (min-sum) algorithm decoding with linear programming decoding, and symbolwise graph-cover decoding links sum-product algorithm decoding with Bethe free energy function minimization at temperature one. In contrast to the Gibbs entropy function, which is a concave function, the Bethe entropy function is in general not concave everywhere. In particular, we show that every code picked from an ensemble of regular low-density parity-check codes with minimum Hamming distance growing (with high probability) linearly with the block length has a Bethe entropy function that is convex in certain regions of its domain.
△ Less
Submitted 9 October, 2012; v1 submitted 30 November, 2010;
originally announced December 2010.
