Showing 1–9 of 9 results for author: Tamir, R

On Correlation Detection and Alignment Recovery of Gaussian Databases

Authors: Ran Tamir

Abstract: In this work, we propose an efficient two-stage algorithm solving a joint problem of correlation detection and partial alignment recovery between two Gaussian databases. Correlation detection is a hypothesis testing problem; under the null hypothesis, the databases are independent, and under the alternate hypothesis, they are correlated, under an unknown row permutation. We develop bounds on the t… ▽ More In this work, we propose an efficient two-stage algorithm solving a joint problem of correlation detection and partial alignment recovery between two Gaussian databases. Correlation detection is a hypothesis testing problem; under the null hypothesis, the databases are independent, and under the alternate hypothesis, they are correlated, under an unknown row permutation. We develop bounds on the type-I and type-II error probabilities, and show that the analyzed detector performs better than a recently proposed detector, at least for some specific parameter choices. Since the proposed detector relies on a statistic, which is a sum of dependent indicator random variables, then in order to bound the type-I probability of error, we develop a novel graph-theoretic technique for bounding the $k$-th order moments of such statistics. When the databases are accepted as correlated, the algorithm also recovers some partial alignment between the given databases. We also propose two more algorithms: (i) One more algorithm for partial alignment recovery, whose reliability and computational complexity are both higher than those of the first proposed algorithm. (ii) An algorithm for full alignment recovery, which has a reduced amount of calculations and a not much lower error probability, when compared to the optimal recovery procedure. △ Less

Submitted 25 May, 2023; v1 submitted 2 November, 2022; originally announced November 2022.

Comments: 43 pages, 20 figures

Fast Convergence to Unanimity in Dense Erdős-Rényi Graphs

Authors: Ran Tamir

Abstract: Majority dynamics on the binomial Erdős-Rényi graph $\mathsf{G}(n,p)$ with $p=λ/\sqrt{n}$ is studied. In this process, each vertex has a state in $\{0,1\}$ and at each round, every vertex adopts the state of the majority of its neighbors, retaining its state in the case of a tie. It was conjectured by Benjamini et al. and proved by Fountoulakis et al. that this process reaches unanimity with high… ▽ More Majority dynamics on the binomial Erdős-Rényi graph $\mathsf{G}(n,p)$ with $p=λ/\sqrt{n}$ is studied. In this process, each vertex has a state in $\{0,1\}$ and at each round, every vertex adopts the state of the majority of its neighbors, retaining its state in the case of a tie. It was conjectured by Benjamini et al. and proved by Fountoulakis et al. that this process reaches unanimity with high probability in at most four rounds. By adding some extra randomness and allowing the underlying graph to be drawn anew in each communication round, we improve on their result and prove that this process reaches consensus in only three communication rounds with probability approaching $1$ as $n$ grows to infinity. We also provide a converse result, showing that three rounds are not only sufficient, but also necessary. △ Less

Submitted 13 October, 2022; v1 submitted 12 October, 2022; originally announced October 2022.

Comments: The introduction has been edited. arXiv admin note: text overlap with arXiv:2104.04996

Error Exponents of the Dirty-Paper and Gel'fand-Pinsker Channels

Authors: Ran Tamir, Neri Merhav

Abstract: We derive various error exponents for communication channels with random states, which are available non-causally at the encoder only. For both the finite-alphabet Gel'fand-Pinsker channel and its Gaussian counterpart, the dirty-paper channel, we derive random coding exponents, error exponents of the typical random codes (TRCs), and error exponents of expurgated codes. For the two channel models,… ▽ More We derive various error exponents for communication channels with random states, which are available non-causally at the encoder only. For both the finite-alphabet Gel'fand-Pinsker channel and its Gaussian counterpart, the dirty-paper channel, we derive random coding exponents, error exponents of the typical random codes (TRCs), and error exponents of expurgated codes. For the two channel models, we analyze some sub-optimal bin-index decoders, which turn out to be asymptotically optimal, at least for the random coding error exponent. For the dirty-paper channel, we show explicitly via a numerical example, that both the error exponent of the TRC and the expurgated exponent strictly improve upon the random coding exponent, at relatively low coding rates, which is a known fact for discrete memoryless channels without random states. We also show that at rates below capacity, the optimal values of the dirty-paper design parameter $α$ in the random coding sense and in the TRC exponent sense are different from one another, and they are both different from the optimal $α$ that is required for attaining the channel capacity. For the Gel'fand-Pinsker channel, we allow for a variable-rate random binning code construction, and prove that the previously proposed maximum penalized mutual information decoder is asymptotically optimal within a given class of decoders, at least for the random coding error exponent. △ Less

Submitted 14 May, 2022; originally announced May 2022.

Entropy Rate Bounds via Second-Order Statistics

Authors: Ran Tamir

Abstract: This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large alphabets. The first bound stems from Gaussian maximum-entropy considerations and depends on the power spectral density (PSD) function of the process. While the… ▽ More This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large alphabets. The first bound stems from Gaussian maximum-entropy considerations and depends on the power spectral density (PSD) function of the process. While the PSD function cannot always be calculated in a closed-form, we also propose a second bound, which merely relies on some finite collection of auto-covariance values of the process. Both of the bounds consist of a one-dimensional integral, while the second bound also consists of a minimization problem over a bounded region, hence they can be efficiently calculated numerically. Examples are also provided to show that the new bounds outperform the standard conditional entropy bound. △ Less

Submitted 10 March, 2022; originally announced March 2022.

Simple Majority Consensus in Networks with Unreliable Communication

Authors: Ran Tamir, Ariel Livshits, Yonatan Shadmi

Abstract: In this work, we analyze the performance of a simple majority-rule protocol solving a fundamental coordination problem in distributed systems - \emph{binary majority consensus}, in the presence of probabilistic message loss. Using probabilistic analysis for a large scale, fully-connected, network of $2n$ agents, we prove that the Simple Majority Protocol (SMP) reaches consensus in only three commu… ▽ More In this work, we analyze the performance of a simple majority-rule protocol solving a fundamental coordination problem in distributed systems - \emph{binary majority consensus}, in the presence of probabilistic message loss. Using probabilistic analysis for a large scale, fully-connected, network of $2n$ agents, we prove that the Simple Majority Protocol (SMP) reaches consensus in only three communication rounds with probability approaching $1$ as $n$ grows to infinity. Moreover, if the difference between the numbers of agents that hold different opinions grows at a rate of $\sqrt{n}$, then the SMP with only two communication rounds attains consensus on the majority opinion of the network, and if this difference grows faster than $\sqrt{n}$, then the SMP reaches consensus on the majority opinion of the network in a single round, with probability converging to $1$ exponentially fast as $n \rightarrow \infty$. We also provide some converse results, showing that these requirements are not only sufficient, but also necessary. △ Less

Submitted 11 April, 2021; originally announced April 2021.

Error Exponents in the Bee Identification Problem

Authors: Ran Tamir, Neri Merhav

Abstract: We derive various error exponents in the bee identification problem under two different decoding rules. Under naïve decoding, which decodes each bee independently of the others, we analyze a general discrete memoryless channel and a relatively wide family of stochastic decoders. Upper and lower bounds to the random coding error exponent are derived and proved to be equal at relatively high coding… ▽ More We derive various error exponents in the bee identification problem under two different decoding rules. Under naïve decoding, which decodes each bee independently of the others, we analyze a general discrete memoryless channel and a relatively wide family of stochastic decoders. Upper and lower bounds to the random coding error exponent are derived and proved to be equal at relatively high coding rates. Then, we propose a lower bound on the error exponent of the typical random code, which improves upon the random coding exponent at low coding rates. We also derive a third bound, which is related to expurgated codes, which turns out to be strictly higher than the other bounds, also at relatively low rates. We show that the universal maximum mutual information decoder is optimal with respect to the typical random code and the expurgated code. Moving further, we derive error exponents under optimal decoding, the relatively wide family of symmetric channels, and the maximum likelihood decoder. We first propose a random coding lower bound, and then, an improved bound which stems from an expurgation process. We show numerically that our second bound strictly improves upon the random coding bound at an intermediate range of coding rates, where a bound derived in a previous work no longer holds. △ Less

Submitted 19 November, 2020; originally announced November 2020.

The MMI Decoder is Asymptotically Optimal for the Typical Random Code and for the Expurgated Code

Authors: Ran Tamir, Neri Merhav

Abstract: We provide two results concerning the optimality of the maximum mutual information (MMI) decoder. First, we prove that the error exponents of the typical random codes under the optimal maximum likelihood (ML) decoder and the MMI decoder are equal. As a corollary to this result, we also show that the error exponents of the expurgated codes under the ML and the MMI decoders are equal. These results… ▽ More We provide two results concerning the optimality of the maximum mutual information (MMI) decoder. First, we prove that the error exponents of the typical random codes under the optimal maximum likelihood (ML) decoder and the MMI decoder are equal. As a corollary to this result, we also show that the error exponents of the expurgated codes under the ML and the MMI decoders are equal. These results strengthen the well known result due to Csiszár and Körner, according to which, these decoders achieve equal random coding error exponents, since the error exponents of the typical random code and the expurgated code are strictly higher than the random coding error exponents, at least at low coding rates. While the universal optimality of the MMI decoder, in the random-coding error exponent sense, is easily proven by commuting the expectation over the channel noise and the expectation over the ensemble, when it comes to typical and expurgated exponents, this commutation can no longer be carried out. Therefore, the proof of the universal optimality of the MMI decoder must be completely different and it turns out to be highly non-trivial. △ Less

Submitted 23 July, 2020; originally announced July 2020.

Trade-offs Between Error Exponents and Excess-Rate Exponents of Typical Slepian-Wolf Codes

Authors: Ran Tamir, Neri Merhav

Abstract: Typical random codes (TRC) in a communication scenario of source coding with side information at the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random binning code ensemble. In this code ensemble, the relatively small type classes of the source are deterministically partitioned into the available bins in a one… ▽ More Typical random codes (TRC) in a communication scenario of source coding with side information at the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random binning code ensemble. In this code ensemble, the relatively small type classes of the source are deterministically partitioned into the available bins in a one-to-one manner. As a consequence, the error probability decreases dramatically. The random binning error exponent and the error exponent of the TRC are derived and proved to be equal to one another in a few important special cases. We show that the performance under optimal decoding can be attained also by certain universal decoders, e.g., the stochastic likelihood decoder with an empirical entropy metric. Moreover, we discuss the trade-offs between the error exponent and the excess-rate exponent for the typical random semi-deterministic code and characterize its optimal rate function. We show that for any pair of correlated information sources, both error and excess-rate probabilities are exponentially vanishing when the blocklength tends to infinity. △ Less

Submitted 27 January, 2021; v1 submitted 17 May, 2020; originally announced May 2020.

Large Deviations Behavior of the Logarithmic Error Probability of Random Codes

Authors: Ran Tamir, Neri Merhav, Nir Weinberger, Albert Guillen i Fabregas

Abstract: This work studies the deviations of the error exponent of the constant composition code ensemble around its expectation, known as the error exponent of the typical random code (TRC). In particular, it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the TRC exponent is exponentially small; upper and lower bounds for this exponent are given, which co… ▽ More This work studies the deviations of the error exponent of the constant composition code ensemble around its expectation, known as the error exponent of the typical random code (TRC). In particular, it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the TRC exponent is exponentially small; upper and lower bounds for this exponent are given, which coincide in some cases. In addition, the probability of randomly drawing a codebook whose error exponent is larger than the TRC exponent is shown to be double-exponentially small; upper and lower bounds to the double-exponential exponent are given. The results suggest that codebooks whose error exponent is larger than the error exponent of the TRC are extremely rare. The key ingredient in the proofs is a new large deviations result of type class enumerators with dependent variables. △ Less

Submitted 20 December, 2019; originally announced December 2019.