Showing 1–50 of 58 results for author: Gohari, A

Impact of Fairness Regulations on Institutions' Policies and Population Qualifications

Authors: Hamidreza Montaseri, Amin Gohari

Abstract: The proliferation of algorithmic systems has fueled discussions surrounding the regulation and control of their social impact. Herein, we consider a system whose primary objective is to maximize utility by selecting the most qualified individuals. To promote demographic parity in the selection algorithm, we consider penalizing discrimination across social groups. We examine conditions under which… ▽ More The proliferation of algorithmic systems has fueled discussions surrounding the regulation and control of their social impact. Herein, we consider a system whose primary objective is to maximize utility by selecting the most qualified individuals. To promote demographic parity in the selection algorithm, we consider penalizing discrimination across social groups. We examine conditions under which a discrimination penalty can effectively reduce disparity in the selection. Additionally, we explore the implications of such a penalty when individual qualifications may evolve over time in response to the imposed penalizing policy. We identify scenarios where the penalty could hinder the natural attainment of equity within the population. Moreover, we propose certain conditions that can counteract this undesirable outcome, thus ensuring fairness. △ Less

Submitted 19 May, 2024; v1 submitted 6 April, 2024; originally announced April 2024.

Comments: Added experiments

On the Inductive Biases of Demographic Parity-based Fair Learning Algorithms

Authors: Haoyu Lei, Amin Gohari, Farzan Farnia

Abstract: Fair supervised learning algorithms assigning labels with little dependence on a sensitive attribute have attracted great attention in the machine learning community. While the demographic parity (DP) notion has been frequently used to measure a model's fairness in training fair classifiers, several studies in the literature suggest potential impacts of enforcing DP in fair learning algorithms. In… ▽ More Fair supervised learning algorithms assigning labels with little dependence on a sensitive attribute have attracted great attention in the machine learning community. While the demographic parity (DP) notion has been frequently used to measure a model's fairness in training fair classifiers, several studies in the literature suggest potential impacts of enforcing DP in fair learning algorithms. In this work, we analytically study the effect of standard DP-based regularization methods on the conditional distribution of the predicted label given the sensitive attribute. Our analysis shows that an imbalanced training dataset with a non-uniform distribution of the sensitive attribute could lead to a classification rule biased toward the sensitive attribute outcome holding the majority of training data. To control such inductive biases in DP-based fair learning, we propose a sensitive attribute-based distributionally robust optimization (SA-DRO) method improving robustness against the marginal distribution of the sensitive attribute. Finally, we present several numerical results on the application of DP-based learning methods to standard centralized and distributed learning problems. The empirical findings support our theoretical results on the inductive biases in DP-based fair learning algorithms and the debiasing effects of the proposed SA-DRO method. △ Less

Submitted 20 June, 2024; v1 submitted 28 February, 2024; originally announced February 2024.

Relative Fractional Independence Number and Its Applications

Authors: Sharareh Alipour, Amin Gohari, Mehrshad Taziki

Abstract: We define the relative fractional independence number of a graph $G$ with respect to another graph $H$, as $$α^*(G|H)=\max_{W}\frac{α(G\boxtimes W)}{α(H\boxtimes W)},$$ where the maximum is taken over all graphs $W$, $G\boxtimes W$ is the strong product of $G$ and $W$, and $α$ denotes the independence number. We give a non-trivial linear program to compute $α^*(G|H)$ and discuss some of its proper… ▽ More We define the relative fractional independence number of a graph $G$ with respect to another graph $H$, as $$α^*(G|H)=\max_{W}\frac{α(G\boxtimes W)}{α(H\boxtimes W)},$$ where the maximum is taken over all graphs $W$, $G\boxtimes W$ is the strong product of $G$ and $W$, and $α$ denotes the independence number. We give a non-trivial linear program to compute $α^*(G|H)$ and discuss some of its properties. We show that $α^*(G|H)\geq \frac{X(G)}{X(H)} \geq \frac{1}{α^*(H|G)},$ where $X(G)$ can be the independence number, the zero-error Shannon capacity, the fractional independence number, the Lovász number, or the Schrijver's or Szegedy's variants of the Lovász number of a graph $G$. This inequality is the first explicit non-trivial upper bound on the ratio of the invariants of two arbitrary graphs, as mentioned earlier, which can also be used to obtain upper or lower bounds for these invariants. As explicit applications, we present new upper bounds for the ratio of the zero-error Shannon capacity of two Cayley graphs and compute new lower bounds on the Shannon capacity of certain Johnson graphs (yielding the exact value of their Haemers number). Moreover, we show that $α^*(G|H)$ can be used to present a stronger version of the well-known No-Homomorphism Lemma. △ Less

Submitted 17 May, 2024; v1 submitted 12 July, 2023; originally announced July 2023.

f-divergences and their applications in lossy compression and bounding generalization error

Authors: Saeed Masiha, Amin Gohari, Mohammad Hossein Yassaee

Abstract: In this paper, we provide three applications for $f$-divergences: (i) we introduce Sanov's upper bound on the tail probability of the sum of independent random variables based on super-modular $f$-divergence and show that our generalized Sanov's bound strictly improves over ordinary one, (ii) we consider the lossy compression problem which studies the set of achievable rates for a given distortion… ▽ More In this paper, we provide three applications for $f$-divergences: (i) we introduce Sanov's upper bound on the tail probability of the sum of independent random variables based on super-modular $f$-divergence and show that our generalized Sanov's bound strictly improves over ordinary one, (ii) we consider the lossy compression problem which studies the set of achievable rates for a given distortion and code length. We extend the rate-distortion function using mutual $f$-information and provide new and strictly better bounds on achievable rates in the finite blocklength regime using super-modular $f$-divergences, and (iii) we provide a connection between the generalization error of algorithms with bounded input/output mutual $f$-information and a generalized rate-distortion problem. This connection allows us to bound the generalization error of learning algorithms using lower bounds on the $f$-rate-distortion function. Our bound is based on a new lower bound on the rate-distortion function that (for some examples) strictly improves over previously best-known bounds. △ Less

Submitted 26 January, 2023; v1 submitted 21 June, 2022; originally announced June 2022.

Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms

Authors: Milad Sefidgaran, Amin Gohari, Gaël Richard, Umut Şimşekli

Abstract: Understanding generalization in modern machine learning settings has been one of the major challenges in statistical learning theory. In this context, recent years have witnessed the development of various generalization bounds suggesting different complexity notions such as the mutual information between the data sample and the algorithm output, compressibility of the hypothesis space, and the fr… ▽ More Understanding generalization in modern machine learning settings has been one of the major challenges in statistical learning theory. In this context, recent years have witnessed the development of various generalization bounds suggesting different complexity notions such as the mutual information between the data sample and the algorithm output, compressibility of the hypothesis space, and the fractal dimension of the hypothesis space. While these bounds have illuminated the problem at hand from different angles, their suggested complexity notions might appear seemingly unrelated, thereby restricting their high-level impact. In this study, we prove novel generalization bounds through the lens of rate-distortion theory, and explicitly relate the concepts of mutual information, compressibility, and fractal dimensions in a single mathematical framework. Our approach consists of (i) defining a generalized notion of compressibility by using source coding concepts, and (ii) showing that the `compression error rate' can be linked to the generalization error both in expectation and with high probability. We show that in the `lossless compression' setting, we recover and improve existing mutual information-based bounds, whereas a `lossy compression' scheme allows us to link generalization to the rate-distortion dimension -- a particular notion of fractal dimension. Our results bring a more unified perspective on generalization and open up several future research directions. △ Less

Submitted 29 June, 2022; v1 submitted 4 March, 2022; originally announced March 2022.

Comments: Accepted for presentation at the Conference on Learning Theory (COLT) 2022

Learning under Distribution Mismatch and Model Misspecification

Authors: Saeed Masiha, Amin Gohari, Mohammad Hossein Yassaee, Mohammad Reza Aref

Abstract: We study learning algorithms when there is a mismatch between the distributions of the training and test datasets of a learning algorithm. The effect of this mismatch on the generalization error and model misspecification are quantified. Moreover, we provide a connection between the generalization error and the rate-distortion theory, which allows one to utilize bounds from the rate-distortion the… ▽ More We study learning algorithms when there is a mismatch between the distributions of the training and test datasets of a learning algorithm. The effect of this mismatch on the generalization error and model misspecification are quantified. Moreover, we provide a connection between the generalization error and the rate-distortion theory, which allows one to utilize bounds from the rate-distortion theory to derive new bounds on the generalization error and vice versa. In particular, the rate-distortion based bound strictly improves over the earlier bound by Xu and Raginsky even when there is no mismatch. We also discuss how "auxiliary loss functions" can be utilized to obtain upper bounds on the generalization error. △ Less

Submitted 10 August, 2022; v1 submitted 10 February, 2021; originally announced February 2021.

Comments: 25 pages, 4 figures

A Strengthened Cutset Upper Bound on the Capacity of the Relay Channel and Applications

Authors: Abbas El Gamal, Amin Gohari, Chandra Nair

Abstract: We develop a new upper bound on the capacity of the relay channel that is tighter than previously known upper bounds. This upper bound is proved using traditional weak converse techniques involving mutual information inequalities and Gallager-type explicit identification of auxiliary random variables. We show that the new upper bound is strictly tighter than all previous bounds for the Gaussian re… ▽ More We develop a new upper bound on the capacity of the relay channel that is tighter than previously known upper bounds. This upper bound is proved using traditional weak converse techniques involving mutual information inequalities and Gallager-type explicit identification of auxiliary random variables. We show that the new upper bound is strictly tighter than all previous bounds for the Gaussian relay channel with non-zero channel gains. When specialized to the relay channel with orthogonal receiver components, the bound resolves a conjecture by Kim on a class of deterministic relay channels. When further specialized to the class of product-form relay channels with orthogonal receiver components, the bound resolves a generalized version of Cover's relay channel problem, recovers the recent upper bound for the Gaussian case by Wu et al., and improves upon the recent bounds for the binary symmetric case by Wu et al. and Barnes et al., which were obtained using non-traditional geometric proof techniques. For the special class of a relay channel with orthogonal receiver components, we develop another upper bound on the capacity which utilizes an auxiliary receiver and show that it is strictly tighter than the bound by Tandon and Ulukus. Finally, we show through the Gaussian relay channel with i.i.d. relay output sequence that the bound with the auxiliary receiver can be strictly tighter than our main bound. △ Less

Submitted 31 March, 2022; v1 submitted 26 January, 2021; originally announced January 2021.

Comments: To appear in the IEEE Transactions on Information Theory. 42 pages, 8 figures

Transmission of a Bit over a Discrete Poisson Channel with Memory

Authors: Niloufar Ahmadypour, Amin Gohari

Abstract: A coding scheme for transmission of a bit maps a given bit to a sequence of channel inputs (called the codeword associated to the transmitted bit). In this paper, we study the problem of designing the best code for a discrete Poisson channel with memory (under peak-power and total-power constraints). The outputs of a discrete Poisson channel with memory are Poisson distributed random variables wit… ▽ More A coding scheme for transmission of a bit maps a given bit to a sequence of channel inputs (called the codeword associated to the transmitted bit). In this paper, we study the problem of designing the best code for a discrete Poisson channel with memory (under peak-power and total-power constraints). The outputs of a discrete Poisson channel with memory are Poisson distributed random variables with a mean comprising of a fixed additive noise and a linear combination of past input symbols. Assuming a maximum-likelihood (ML) decoder, we search for a codebook that has the smallest possible error probability. This problem is challenging because error probability of a code does not have a closed-form analytical expression. For the case of having only a total-power constraint, the optimal code structure is obtained, provided that the blocklength is greater than the memory length of the channel. For the case of having only a peak-power constraint, the optimal code is derived for arbitrary memory and blocklength in the high-power regime. For the case of having both the peak-power and total-power constraints, the optimal code is derived for memoryless Poisson channels when both the total-power and the peak-power bounds are large. △ Less

Submitted 14 April, 2021; v1 submitted 11 November, 2020; originally announced November 2020.

An Upper Bound on Secret Key Rates for General Multiterminal Wiretap Channels

Authors: Amin Gohari, Gerhard Kramer

Abstract: An upper bound is derived on the secret key rates of a general multiterminal wiretap channel. The bound unifies and generalizes some of the previously known bounds. Additionally, a multivariate dependence balance bound is introduced that is of independent interest. An upper bound is derived on the secret key rates of a general multiterminal wiretap channel. The bound unifies and generalizes some of the previously known bounds. Additionally, a multivariate dependence balance bound is introduced that is of independent interest. △ Less

Submitted 6 February, 2023; v1 submitted 30 September, 2020; originally announced September 2020.

Multi-Way Number Partitioning: an Information-Theoretic View

Authors: Niloufar Ahmadypour, Amin Gohari

Abstract: The number partitioning problem is the problem of partitioning a given list of numbers into multiple subsets so that the sum of the numbers in each subset are as nearly equal as possible. We introduce two closely related notions of the "most informative" and "most compressible" partitions. Most informative partitions satisfy a principle of optimality property. We also give an exact algorithm (base… ▽ More The number partitioning problem is the problem of partitioning a given list of numbers into multiple subsets so that the sum of the numbers in each subset are as nearly equal as possible. We introduce two closely related notions of the "most informative" and "most compressible" partitions. Most informative partitions satisfy a principle of optimality property. We also give an exact algorithm (based on Huffman coding) with a running time of O(nlog(n)) in input size n to find the most compressible partition. △ Less

Submitted 6 September, 2020; originally announced September 2020.

An Analytical Model for Molecular Communication over a Non-linear Reaction-Diffusion Medium

Authors: Hamidreza Abin, Amin Gohari, Masoumeh Nasiri-Kenari

Abstract: One of the main challenges in diffusion-based molecular communication is dealing with the non-linearity of reaction-diffusion chemical equations. While numerical methods can be used to solve these equations, a change in the input signals or the parameters of the medium requires one to redo the simulations. This makes it difficult to design modulation schemes and practically impossible to prove the… ▽ More One of the main challenges in diffusion-based molecular communication is dealing with the non-linearity of reaction-diffusion chemical equations. While numerical methods can be used to solve these equations, a change in the input signals or the parameters of the medium requires one to redo the simulations. This makes it difficult to design modulation schemes and practically impossible to prove the optimality of a given transmission strategy. In this paper, we provide an analytical technique for modeling the non-linearity of chemical reaction equations based on the perturbation method. The perturbation method expresses the solution in terms of an infinite power series. An approximate solution can be found by kee** the leading terms of the power series. The approximate solution is shown to track the true solution if either the simulation time interval or the reaction rate is sufficiently small. Approximate solutions for long time intervals are also discussed. An illustrative example is given. For this example, it is shown that when the reaction rate (or the total time interval) is low, instead of using a continuous release waveform, it is optimal for the transmitters to release molecules at two time instances. △ Less

Submitted 2 June, 2021; v1 submitted 28 May, 2020; originally announced May 2020.

Playing Games with Bounded Entropy: Convergence Rate and Approximate Equilibria

Authors: Mehrdad Valizadeh, Amin Gohari

Abstract: We consider zero-sum repeated games in which the players are restricted to strategies that require only a limited amount of randomness. Let $v_n$ be the max-min value of the $n$ stage game; previous works have characterized $\lim_{n\rightarrow\infty}v_n$, i.e., the long-run max-min value. Our first contribution is to study the convergence rate of $v_n$ to its limit. To this end, we provide a new t… ▽ More We consider zero-sum repeated games in which the players are restricted to strategies that require only a limited amount of randomness. Let $v_n$ be the max-min value of the $n$ stage game; previous works have characterized $\lim_{n\rightarrow\infty}v_n$, i.e., the long-run max-min value. Our first contribution is to study the convergence rate of $v_n$ to its limit. To this end, we provide a new tool for simulation of a source (target source) from another source (coin source). Considering the total variation distance as the measure of precision, this tool offers an upper bound for the precision of simulation, which is vanishing exponentially in the difference of Rényi entropies of the coin and target sources. In the second part of paper, we characterize the set of all approximate Nash equilibria achieved in long run. It turns out that this set is in close relation with the long-run max-min value. △ Less

Submitted 10 February, 2019; originally announced February 2019.

A Correlation Measure Based on Vector-Valued $L_p$-Norms

Authors: Mohammad Mahdi Mojahedian, Salman Beigi, Amin Gohari, Mohammad Hossein Yassaee, Mohammad Reza Aref

Abstract: In this paper, we introduce a new measure of correlation for bipartite quantum states. This measure depends on a parameter $α$, and is defined in terms of vector-valued $L_p$-norms. The measure is within a constant of the exponential of $α$-Rényi mutual information, and reduces to the trace norm (total variation distance) for $α=1$. We will prove some decoupling type theorems in terms of this meas… ▽ More In this paper, we introduce a new measure of correlation for bipartite quantum states. This measure depends on a parameter $α$, and is defined in terms of vector-valued $L_p$-norms. The measure is within a constant of the exponential of $α$-Rényi mutual information, and reduces to the trace norm (total variation distance) for $α=1$. We will prove some decoupling type theorems in terms of this measure of correlation, and present some applications in privacy amplification as well as in bounding the random coding exponents. In particular, we establish a bound on the secrecy exponent of the wiretap channel (under the total variation metric) in terms of the $α$-Rényi mutual information according to \emph{Csiszár's proposal}. △ Less

Submitted 21 May, 2018; originally announced May 2018.

Comments: 41 pages

Diffusion Based Molecular Communication with Limited Molecule Production Rate

Authors: Hamid G. Bafghi, Amin Gohari, Mahtab Mirmohseni, Masoumeh Nasiri-Kenari

Abstract: This paper studies the impact of a transmitter's molecule generation process on the capacity of a concentration based Molecular Communication (MC) system. Constraints caused by the molecule generation process affect the availability of the molecules at the transmitter. The transmitter has a storage of molecules, and should decide whether to release or save the currently produced molecules. As a re… ▽ More This paper studies the impact of a transmitter's molecule generation process on the capacity of a concentration based Molecular Communication (MC) system. Constraints caused by the molecule generation process affect the availability of the molecules at the transmitter. The transmitter has a storage of molecules, and should decide whether to release or save the currently produced molecules. As a result, the MC system has conceptual connections with energy harvesting systems. In this paper, we consider two scenarios on the propagation channel. The first scenario assumes a channel with no Inter-symbol Interference (ISI), \emph{i.e.,} a memoryless channel. We derive bounds on the capacity of the MC system in this scenario. The second scenario assumes the MC network with ISI, in which the output of the channel depends on the history of released molecules in the previous time-slots. Based on the assumptions that either the transmitter or the receiver knows the channel statistics, we compute a lower bound on the channel capacity. △ Less

Submitted 25 February, 2018; originally announced February 2018.

Coding for Positive Rate in the Source Model Key Agreement Problem

Authors: Amin Gohari, Onur Günlü, Gerhard Kramer

Abstract: A two-party key agreement problem with public discussion, known as the source model problem, is considered. By relating key agreement to hypothesis testing, a new coding scheme is developed that yields a sufficient condition to achieve a positive secret-key (SK) rate in terms of Rényi divergence. The merits of this coding scheme are illustrated by applying it to an erasure model for Eve's side inf… ▽ More A two-party key agreement problem with public discussion, known as the source model problem, is considered. By relating key agreement to hypothesis testing, a new coding scheme is developed that yields a sufficient condition to achieve a positive secret-key (SK) rate in terms of Rényi divergence. The merits of this coding scheme are illustrated by applying it to an erasure model for Eve's side information, and by deriving an upper bound on Eve's erasure probabilities for which the SK capacity is zero. This bound strictly improves on the best known single-letter lower bound on the SK capacity. Moreover, the bound is tight when Alice's or Bob's source is binary, which extends a previous result for a doubly symmetric binary source. The results motivate a new measure for the correlation between two random variables, which is of independent interest. △ Less

Submitted 22 July, 2020; v1 submitted 15 September, 2017; originally announced September 2017.

Comments: To appear in the IEEE Transactions on Information Theory

How Compressible are Innovation Processes?

Authors: Hamid Ghourchian, Arash Amini, Amin Gohari

Abstract: The sparsity and compressibility of finite-dimensional signals are of great interest in fields such as compressed sensing. The notion of compressibility is also extended to infinite sequences of i.i.d. or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this work, we use the entropy measure to study the compressibility of continuous-domain innovation… ▽ More The sparsity and compressibility of finite-dimensional signals are of great interest in fields such as compressed sensing. The notion of compressibility is also extended to infinite sequences of i.i.d. or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this work, we use the entropy measure to study the compressibility of continuous-domain innovation processes (alternatively known as white noise). Specifically, we define such a measure as the entropy limit of the doubly quantized (time and amplitude) process. This provides a tool to compare the compressibility of various innovation processes. It also allows us to identify an analogue of the concept of "entropy dimension" which was originally defined by Rényi for random variables. Particular attention is given to stable and impulsive Poisson innovation processes. Here, our results recognize Poisson innovations as the more compressible ones with an entropy measure far below that of stable innovations. While this result departs from the previous knowledge regarding the compressibility of fat-tailed distributions, our entropy measure ranks stable innovations according to their tail decay. △ Less

Submitted 21 January, 2018; v1 submitted 28 March, 2017; originally announced March 2017.

Comments: 58 pages

Existence and Continuity of Differential Entropy for a Class of Distributions

Authors: Hamid Ghourchian, Amin Gohari, Arash Amini

Abstract: In this paper, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of this class is that the requirements could be readily verified for a given distribution. In this paper, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of this class is that the requirements could be readily verified for a given distribution. △ Less

Submitted 28 March, 2017; originally announced March 2017.

Comments: 4 pages

Journal ref: IEEE Communications Letters, 21 (7), 1469-1472 (2017)

Playing Games with Bounded Entropy

Authors: Mehrdad Valizadeh, Amin Gohari

Abstract: In this paper, we consider zero-sum repeated games in which the maximizer is restricted to strategies requiring no more than a limited amount of randomness. Particularly, we analyze the maxmin payoff of the maximizer in two models: the first model forces the maximizer to randomize her action in each stage just by conditioning her decision to outcomes of a given sequence of random source, whereas,… ▽ More In this paper, we consider zero-sum repeated games in which the maximizer is restricted to strategies requiring no more than a limited amount of randomness. Particularly, we analyze the maxmin payoff of the maximizer in two models: the first model forces the maximizer to randomize her action in each stage just by conditioning her decision to outcomes of a given sequence of random source, whereas, in the second model, the maximizer is a team of players who are free to privately randomize their corresponding actions but do not have access to any explicit source of shared randomness needed for cooperation. The works of Gossner and Vieille, and Gossner and Tomala adopted the method of types to establish their results; however, we utilize the idea of random hashing which is the core of randomness extractors in the information theory literature. In addition, we adopt the well-studied tool of simulation of a source from another source. By utilizing these tools, we are able to simplify the prior results and generalize them as well. We characterize the maxmin payoff of the maximizer in the repeated games under study. Particularly, the maxmin payoff of the first model is fully described by the function $J(h)$ which is the maximum payoff that the maximizer can secure in a one-shot game by choosing mixed strategies of entropy at most $h$. In the second part of the paper, we study the computational aspects of $J(h)$. We offer three explicit lower bounds on the entropy-payoff trade-off curve. To do this, we provide and utilize new results for the set of distributions that guarantee a certain payoff for Alice. In particular, we study how this set of distributions shrinks as we increase the security level. While the use of total variation distance is common in game theory, our derivation indicates the suitability of utilizing the Renyi-divergence of order two. △ Less

Submitted 10 October, 2018; v1 submitted 19 February, 2017; originally announced February 2017.

On the Capacity of a Class of Signal-Dependent Noise Channels

Authors: Hamid Ghourchian, Gholamali Aminian, Amin Gohari, Mahtab Mirmohseni, Masoumeh Nasiri-Kenari

Abstract: In some applications, the variance of additive measurement noise depends on the signal that we aim to measure. For instance, additive Gaussian signal-dependent noise (AGSDN) channel models are used in molecular and optical communication. Herein we provide lower and upper bounds on the capacity of additive signal-dependent noise (ASDN) channels. The idea of the first lower bound is the extension of… ▽ More In some applications, the variance of additive measurement noise depends on the signal that we aim to measure. For instance, additive Gaussian signal-dependent noise (AGSDN) channel models are used in molecular and optical communication. Herein we provide lower and upper bounds on the capacity of additive signal-dependent noise (ASDN) channels. The idea of the first lower bound is the extension of the majorization inequality, and for the second one, it uses some calculations based on the fact that $h(Y) > h (Y|Z)$. Both of them are valid for all additive signal-dependent noise (ASDN) channels defined in the paper. The upper bound is based on a previous idea of the authors ("symmetric relative entropy") and is used for the additive Gaussian signal-dependent noise (AGSDN) channels. These bounds indicate that in ASDN channels (unlike the classical AWGN channels), the capacity does not necessarily become larger by making the variance function of the noise smaller. We also provide sufficient conditions under which the capacity becomes infinity. This is complemented by a number of conditions that imply capacity is finite and a unique capacity achieving measure exists (in the sense of the output measure). △ Less

Submitted 2 June, 2017; v1 submitted 12 February, 2017; originally announced February 2017.

Comments: 34 pages, 3 figures

Information Theory of Molecular Communication: Directions and Challenges

Authors: Amin Gohari, Mahtab Mirmohseni, Masoumeh Nasiri-Kenari

Abstract: Molecular Communication (MC) is a communication strategy that uses molecules as carriers of information, and is widely used by biological cells. As an interdisciplinary topic, it has been studied by biologists, communication theorists and a growing number of information theorists. This paper aims to specifically bring MC to the attention of information theorists. To do this, we first highlight the… ▽ More Molecular Communication (MC) is a communication strategy that uses molecules as carriers of information, and is widely used by biological cells. As an interdisciplinary topic, it has been studied by biologists, communication theorists and a growing number of information theorists. This paper aims to specifically bring MC to the attention of information theorists. To do this, we first highlight the unique mathematical challenges of studying the capacity of molecular channels. Addressing these problems require use of known, or development of new mathematical tools. Toward this goal, we review a subjective selection of the existing literature on information theoretic aspect of molecular communication. The emphasis here is on the mathematical techniques used, rather than on the setup or modeling of a specific paper. Finally, as an example, we propose a concrete information theoretic problem that was motivated by our study of molecular communication. △ Less

Submitted 10 December, 2016; originally announced December 2016.

Comments: Accepted for publication in the IEEE Transactions on Molecular, Biological, and Multi-Scale Communications as a Shannon Centennial Special Issue

Phi-Entropic Measures of Correlation

Authors: Salman Beigi, Amin Gohari

Abstract: A measure of correlation is said to have the tensorization property if it is unchanged when computed for i.i.d.\ copies. More precisely, a measure of correlation between two random variables $(X, Y)$ denoted by $ρ(X, Y)$, has the tensorization property if $ρ(X^n, Y^n)=ρ(X, Y)$ where $(X^n, Y^n)$ is $n$ i.i.d.\ copies of $(X, Y)$.Two well-known examples of such measures are the maximal correlation… ▽ More A measure of correlation is said to have the tensorization property if it is unchanged when computed for i.i.d.\ copies. More precisely, a measure of correlation between two random variables $(X, Y)$ denoted by $ρ(X, Y)$, has the tensorization property if $ρ(X^n, Y^n)=ρ(X, Y)$ where $(X^n, Y^n)$ is $n$ i.i.d.\ copies of $(X, Y)$.Two well-known examples of such measures are the maximal correlation and the hypercontractivity ribbon (HC~ribbon). We show that the maximal correlation and HC ribbons are special cases of $Φ$-ribbon, defined in this paper for any function $Φ$ from a class of convex functions ($Φ$-ribbon reduces to HC~ribbon and the maximal correlation for special choices of $Φ$). Any $Φ$-ribbon is shown to be a measures of correlation with the tensorization property. We show that the $Φ$-ribbon also characterizes the $Φ$-strong data processing inequality constant introduced by Raginsky. We further study the $Φ$-ribbon for the choice of $Φ(t)=t^2$ and introduce an equivalent characterization of this ribbon. △ Less

Submitted 4 November, 2016; originally announced November 2016.

On the Equivalency of Reliability and Security Metrics for Wireline Networks

Authors: Mohammad Mahdi Mojahedian, Amin Gohari, Mohammad Reza Aref

Abstract: In this paper, we show the equivalency of weak and strong secrecy conditions for a large class of secure network coding problems. When we restrict to linear operations, we show the equivalency of "perfect secrecy and zero-error constraints" with "weak secrecy and $ε$-error constraints". In this paper, we show the equivalency of weak and strong secrecy conditions for a large class of secure network coding problems. When we restrict to linear operations, we show the equivalency of "perfect secrecy and zero-error constraints" with "weak secrecy and $ε$-error constraints". △ Less

Submitted 15 September, 2016; originally announced September 2016.

Comments: 21 pages, 2 figures

On Medium Chemical Reaction in Diffusion-Based Molecular Communication: a Two-Way Relaying Example

Authors: Maryam Farahnak-Ghazani, Gholamali Aminian, Mahtab Mirmohseni, Amin Gohari, Masoumeh Nasiri-Kenari

Abstract: Chemical reactions are a prominent feature of molecular communication (MC) systems, with no direct parallels in wireless communications. While chemical reactions may be used inside the transmitter nodes, receiver nodes or the communication medium, we focus on its utility in the medium in this paper. Such chemical reactions can be used to perform computation over the medium as molecules diffuse and… ▽ More Chemical reactions are a prominent feature of molecular communication (MC) systems, with no direct parallels in wireless communications. While chemical reactions may be used inside the transmitter nodes, receiver nodes or the communication medium, we focus on its utility in the medium in this paper. Such chemical reactions can be used to perform computation over the medium as molecules diffuse and react with each other (physical-layer computation). We propose the use of chemical reactions for the following purposes: (i) to reduce signal-dependent observation noise of receivers by reducing the signal density, (ii) to realize molecular physical-layer network coding (molecular PNC) by performing the natural XOR operation inside the medium, and (iii) to reduce the inter-symbol interference (ISI) of other transmitters by canceling out the remaining molecules from previous transmissions. To make the ideas formal, we consider an explicit two-way relaying example with a transparent receiver (which has a signal-dependent noise). The proposed ideas are used to define a modulation scheme (which we call the PNC scheme). We compare the PNC with a previously proposed scheme for this problem where the XOR operation is performed at the relay node (using a molecular logic gate). We call the latter, the straightforward network coding (SNC). It is observed that in addition to the simplicity of the proposed PNC scheme, it outperforms the SNC scheme especially when we consider ISI. △ Less

Submitted 24 April, 2018; v1 submitted 19 April, 2016; originally announced April 2016.

Comments: 32 pages, 6 figures

Type Based Sign Modulation and its Application for ISI mitigation in Molecular Communication

Authors: Reza Mosayebi, Amin Gohari, Mahtab Mirmohseni, Masoumeh Nasiri Kenari

Abstract: An important challenge in design of modulation schemes for molecular communication is positivity of the transmission signal (only a positive concentration of molecules can be released in the environment). This restriction makes handling of the InterSymbol Interference (ISI) a challenge for molecular communication. Previous works have proposed use of chemical reactions to remove molecules from the… ▽ More An important challenge in design of modulation schemes for molecular communication is positivity of the transmission signal (only a positive concentration of molecules can be released in the environment). This restriction makes handling of the InterSymbol Interference (ISI) a challenge for molecular communication. Previous works have proposed use of chemical reactions to remove molecules from the environment, and to effectively simulate negative signals. However, the differential equation describing a diffusion-reaction process is non-linear. This precludes the possibility of using Fourier transform tools. In this paper, a solution for simulating negative signals based on the diffusion-reaction channel model is proposed. While the proposed solution does not exploit the full degrees of freedom available for signaling in a diffusion-reaction process, but its end-to-end system is a linear channel and amenable to Fourier transform analysis. Based on our solution, a modulation scheme and a precoder are introduced and shown to have a significant reduction in error probability compared to previous modulation schemes such as CSK and MCSK. The effect of various imperfections (such as quantization error) on the communication system performance are studied. △ Less

Submitted 14 December, 2016; v1 submitted 6 March, 2016; originally announced March 2016.

On the Monotone Measure of Correlation

Authors: Omid Etesami, Amin Gohari

Abstract: Based on the notion of maximal correlation, Kimeldorf, May and Sampson (1980) introduce a measure of correlation between two random variables, called the "concordant monotone correlation" (CMC). We revisit, generalize and prove new properties of this measure of correlation. It is shown that CMC captures various types of correlation detected in measures of rank correlation like the Kendall tau corr… ▽ More Based on the notion of maximal correlation, Kimeldorf, May and Sampson (1980) introduce a measure of correlation between two random variables, called the "concordant monotone correlation" (CMC). We revisit, generalize and prove new properties of this measure of correlation. It is shown that CMC captures various types of correlation detected in measures of rank correlation like the Kendall tau correlation. We show that the CMC satisfies the data processing and tensorization properties (that make ordinary maximal correlation applicable to problems in information theory). Furthermore, CMC is shown to be intimately related to the FKG inequality. Furthermore, a combinatorical application of CMC is given for which we do not know of another method to derive its result. Finally, we study the problem of the complexity of the computation of the CMC, which is a non-convex optimization problem with local maximas. We give a simple but exponential-time algorithm that is guaranteed to output the exact value of the generalized CMC. △ Less

Submitted 22 June, 2016; v1 submitted 13 November, 2015; originally announced November 2015.

Journal ref: IEEE Communication Letters, 2015, Volume 20, Issue 1, Pages 17-20

High Probability Guarantees in Repeated Games: Theory and Applications in Information Theory

Authors: Payam Delgosha, Amin Gohari, Mohammad Akbarpour

Abstract: We introduce a "high probability" framework for repeated games with incomplete information. In our non-equilibrium setting, players aim to guarantee a certain payoff with high probability, rather than in expected value. We provide a high probability counterpart of the classical result of Mertens and Zamir for the zero-sum repeated games. Any payoff that can be guaranteed with high probability can… ▽ More We introduce a "high probability" framework for repeated games with incomplete information. In our non-equilibrium setting, players aim to guarantee a certain payoff with high probability, rather than in expected value. We provide a high probability counterpart of the classical result of Mertens and Zamir for the zero-sum repeated games. Any payoff that can be guaranteed with high probability can be guaranteed in expectation, but the reverse is not true. Hence, unlike the average payoff case where the payoff guaranteed by each player is the negative of the payoff by the other player, the two guaranteed payoffs would differ in the high probability framework. One motivation for this framework comes from information transmission systems, where it is customary to formulate problems in terms of asymptotically vanishing probability of error. An application of our results to a class of compound arbitrarily varying channels is given. △ Less

Submitted 28 September, 2015; originally announced September 2015.

On ISI-free Modulations for Diffusion based Molecular Communication

Authors: Hamidreza Arjmandi, Mohammad Movahednasab, Amin Gohari, Mahtab Mirmohseni, Masoumeh Nasiri Kenari, Faramarz Fekri

Abstract: A diffusion molecular channel is a channel with memory, as molecules released into the medium hit the receptors after a random delay. Coding over the diffusion channel is performed by choosing the type, intensity, or the released time of molecules diffused in the environment over time. To avoid intersymbol interference (ISI), molecules of the same type should be released at time instances that are… ▽ More A diffusion molecular channel is a channel with memory, as molecules released into the medium hit the receptors after a random delay. Coding over the diffusion channel is performed by choosing the type, intensity, or the released time of molecules diffused in the environment over time. To avoid intersymbol interference (ISI), molecules of the same type should be released at time instances that are sufficiently far apart. This ensures that molecules of a previous transmission are faded in the environment, before molecules of the same type are reused for signaling. In this paper, we consider ISI-free time-slotted modulation schemes. The maximum reliable transmission rate for these modulations is given by the constrained coding capacity of the graph that represents the permissible transmission sequences. However, achieving the constrained coding capacity requires long blocklengths and delays at the decoder, making it impractical for simple nanomachines. The main contribution of this paper is to consider modulations with small delay (short blocklength) and show that they get very close to constrained coding capacity. △ Less

Submitted 11 August, 2015; originally announced August 2015.

Zero Error Coordination

Authors: Mahed Abroshan, Amin Gohari, Sidharth Jaggi

Abstract: In this paper, we consider a zero error coordination problem wherein the nodes of a network exchange messages to be able to perfectly coordinate their actions with the individual observations of each other. While previous works on coordination commonly assume an asymptotically vanishing error, we assume exact, zero error coordination. Furthermore, unlike previous works that employ the empirical or… ▽ More In this paper, we consider a zero error coordination problem wherein the nodes of a network exchange messages to be able to perfectly coordinate their actions with the individual observations of each other. While previous works on coordination commonly assume an asymptotically vanishing error, we assume exact, zero error coordination. Furthermore, unlike previous works that employ the empirical or strong notions of coordination, we define and use a notion of set coordination. This notion of coordination bears similarities with the empirical notion of coordination. We observe that set coordination, in its special case of two nodes with a one-way communication link is equivalent with the "Hide and Seek" source coding problem of McEliece and Posner. The Hide and Seek problem has known intimate connections with graph entropy, rate distortion theory, Renyi mutual information and even error exponents. Other special cases of the set coordination problem relate to Witsenhausen's zero error rate and the distributed computation problem. These connections motivate a better understanding of set coordination, its connections with empirical coordination, and its study in more general setups. This paper takes a first step in this direction by proving new results for two node networks. △ Less

Submitted 5 May, 2015; originally announced May 2015.

Perfectly Secure Index Coding

Authors: Mohammad Mahdi Mojahedian, Mohammad Reza Aref, Amin Gohari

Abstract: In this paper, we investigate the index coding problem in the presence of an eavesdropper. Messages are to be sent from one transmitter to a number of legitimate receivers who have side information about the messages, and share a set of secret keys with the transmitter. We assume perfect secrecy, meaning that the eavesdropper should not be able to retrieve any information about the message set. We… ▽ More In this paper, we investigate the index coding problem in the presence of an eavesdropper. Messages are to be sent from one transmitter to a number of legitimate receivers who have side information about the messages, and share a set of secret keys with the transmitter. We assume perfect secrecy, meaning that the eavesdropper should not be able to retrieve any information about the message set. We study the minimum key lengths for zero-error and perfectly secure index coding problem. On one hand, this problem is a generalization of the index coding problem (and thus a difficult one). On the other hand, it is a generalization of the Shannon's cipher system. We show that a generalization of Shannon's one-time pad strategy is optimal up to a multiplicative constant, meaning that it obtains the entire boundary of the cone formed by looking at the secure rate region from the origin. Finally, we consider relaxation of the perfect secrecy and zero-error constraints to weak secrecy and asymptotically vanishing probability of error, and provide a secure version of the result, obtained by Langberg and Effros, on the equivalence of zero-error and $ε$-error regions in the conventional index coding problem. △ Less

Submitted 30 December, 2015; v1 submitted 17 April, 2015; originally announced April 2015.

Comments: 25 pages, 5 figures, submitted to the IEEE Transactions on Information Theory

On the Duality of Additivity and Tensorization

Authors: Salman Beigi, Amin Gohari

Abstract: A function is said to be additive if, similar to mutual information, expands by a factor of $n$, when evaluated on $n$ i.i.d. repetitions of a source or channel. On the other hand, a function is said to satisfy the tensorization property if it remains unchanged when evaluated on i.i.d. repetitions. Additive rate regions are of fundamental importance in network information theory, serving as capaci… ▽ More A function is said to be additive if, similar to mutual information, expands by a factor of $n$, when evaluated on $n$ i.i.d. repetitions of a source or channel. On the other hand, a function is said to satisfy the tensorization property if it remains unchanged when evaluated on i.i.d. repetitions. Additive rate regions are of fundamental importance in network information theory, serving as capacity regions or upper bounds thereof. Tensorizing measures of correlation have also found applications in distributed source and channel coding problems as well as the distribution simulation problem. Prior to our work only two measures of correlation, namely the hypercontractivity ribbon and maximal correlation (and their derivatives), were known to have the tensorization property. In this paper, we provide a general framework to obtain a region with the tensorization property from any additive rate region. We observe that hypercontractivity ribbon indeed comes from the dual of the rate region of the Gray-Wyner source coding problem, and generalize it to the multipartite case. Then we define other measures of correlation with similar properties from other source coding problems. We also present some applications of our results. △ Less

Submitted 4 November, 2016; v1 submitted 3 February, 2015; originally announced February 2015.

Comments: 31 pages, 3 figures

Deterministic Randomness Extraction from Generalized and Distributed Santha-Vazirani Sources

Authors: Salman Beigi, Omid Etesami, Amin Gohari

Abstract: A Santha-Vazirani (SV) source is a sequence of random bits where the conditional distribution of each bit, given the previous bits, can be partially controlled by an adversary. Santha and Vazirani show that deterministic randomness extraction from these sources is impossible. In this paper, we study the generalization of SV sources for non-binary sequences. We show that unlike the binary case, det… ▽ More A Santha-Vazirani (SV) source is a sequence of random bits where the conditional distribution of each bit, given the previous bits, can be partially controlled by an adversary. Santha and Vazirani show that deterministic randomness extraction from these sources is impossible. In this paper, we study the generalization of SV sources for non-binary sequences. We show that unlike the binary case, deterministic randomness extraction in the generalized case is sometimes possible. We present a necessary condition and a sufficient condition for the possibility of deterministic randomness extraction. These two conditions coincide in "non-degenerate" cases. Next, we turn to a distributed setting. In this setting the SV source consists of a random sequence of pairs $(a_1, b_1), (a_2, b_2), \ldots$ distributed between two parties, where the first party receives $a_i$'s and the second one receives $b_i$'s. The goal of the two parties is to extract common randomness without communication. Using the notion of maximal correlation, we prove a necessary condition and a sufficient condition for the possibility of common randomness extraction from these sources. Based on these two conditions, the problem of common randomness extraction essentially reduces to the problem of randomness extraction from (non-distributed) SV sources. This result generalizes results of Gács and Körner, and Witsenhausen about common randomness extraction from i.i.d. sources to adversarial sources. △ Less

Submitted 20 December, 2014; originally announced December 2014.

Adaptive Molecule Transmission Rate for Diffusion Based Molecular Communication

Authors: Mohammad Movahednasab, Mehdi Soleimanifar, Amin Gohari, Masoumeh Nasiri Kenari, Urbashi Mitra

Abstract: In this paper, a simple memory limited transmitter for molecular communication is proposed, in which information is encoded in the diffusion rate of the molecules. Taking advantage of memory, the proposed transmitter reduces the ISI problem by properly adjusting its diffusion rate. The error probability of the proposed scheme is derived and the result is compared with the lower bound on error prob… ▽ More In this paper, a simple memory limited transmitter for molecular communication is proposed, in which information is encoded in the diffusion rate of the molecules. Taking advantage of memory, the proposed transmitter reduces the ISI problem by properly adjusting its diffusion rate. The error probability of the proposed scheme is derived and the result is compared with the lower bound on error probability of the optimum transmitter. It is shown that the performance of introduced transmitter is near optimal (under certain simplifications). Simplicity is the key feature of the presented communication system: the transmitter follows a simple rule, the receiver is a simple threshold decoder and only one type of molecule is used to convey the information. △ Less

Submitted 29 October, 2014; originally announced October 2014.

Capacity of Diffusion based Molecular Communication Networks over LTI-Poisson Channels

Authors: Hamidreza Arjmandi, Gholamali Aminian, Amin Gohari, Masoumeh Nasiri Kenari, Urbashi Mitra

Abstract: In this paper, the capacity of a diffusion based molecular communication network under the model of a Linear Time Invarient-Poisson (LTI-Poisson) channel is studied. Introduced in the context of molecular communication, the LTI-Poisson model is a natural extension of the conventional memoryless Poisson channel to include memory. Exploiting prior art on linear ISI channels, a computable finite-lett… ▽ More In this paper, the capacity of a diffusion based molecular communication network under the model of a Linear Time Invarient-Poisson (LTI-Poisson) channel is studied. Introduced in the context of molecular communication, the LTI-Poisson model is a natural extension of the conventional memoryless Poisson channel to include memory. Exploiting prior art on linear ISI channels, a computable finite-letter characterization of the capacity of single-hop LTI-Poisson networks is provided. Then, the problem of finding more explicit bounds on the capacity is examined, where lower and upper bounds for the point to point case are provided. Furthermore, an approach for bounding mutual information in the low SNR regime using the symmetrized KL divergence is introduced and its applicability to Poisson channels is shown. To best of our knowledge, the first non-trivial upper bound on the capacity of Poisson channel with a maximum transmission constraint in the low SNR regime is found. Numerical results show that the proposed upper bound is of the same order as the capacity in the low SNR regime. △ Less

Submitted 16 October, 2014; v1 submitted 15 October, 2014; originally announced October 2014.

Monotone Measures for Non-Local Correlations

Authors: Salman Beigi, Amin Gohari

Abstract: Non-locality is the phenomenon of observing strong correlations among the outcomes of local measurements of a multipartite physical system. No-signaling boxes are the abstract objects for studying non-locality, and wirings are local operations on the space of no-signaling boxes. This means that, no matter how non-local the nature is, the set of physical non-local correlations must be closed under… ▽ More Non-locality is the phenomenon of observing strong correlations among the outcomes of local measurements of a multipartite physical system. No-signaling boxes are the abstract objects for studying non-locality, and wirings are local operations on the space of no-signaling boxes. This means that, no matter how non-local the nature is, the set of physical non-local correlations must be closed under wirings. Then, one approach to identify the non-locality of nature is to characterize closed sets of non-local correlations. Although non-trivial examples of wirings of no-signaling boxes are known, there is no systematic way to study wirings. In particular, given a set of no-signaling boxes, we do not know a general method to prove that it is closed under wirings. In this paper, we propose the first general method to construct such closed sets of non-local correlations. We show that a well-known measure of correlation, called maximal correlation, when appropriately defined for non-local correlations, is monotonically decreasing under wirings. This establishes a conjecture about the impossibility of simulating isotropic boxes from each other, implying the existence of a continuum of closed sets of non-local boxes under wirings. To prove our main result, we introduce some mathematical tools that may be of independent interest: we define a notion of maximal correlation ribbon as a generalization of maximal correlation, and provide a connection between it and a known object called hypercontractivity ribbon; we show that these two ribbons are monotone under wirings too. △ Less

Submitted 11 January, 2017; v1 submitted 12 September, 2014; originally announced September 2014.

Comments: 39 pages, 1 table, 3 figures, final version

Journal ref: IEEE Transactions on Information Theory 61(9), 5185-5208 (2015)

The Value of Help Bits in Randomized and Average-Case Complexity

Authors: Salman Beigi, Omid Etesami, Amin Gohari

Abstract: "Help bits" are some limited trusted information about an instance or instances of a computational problem that may reduce the computational complexity of solving that instance or instances. In this paper, we study the value of help bits in the settings of randomized and average-case complexity. Amir, Beigel, and Gasarch (1990) show that for constant $k$, if $k$ instances of a decision problem c… ▽ More "Help bits" are some limited trusted information about an instance or instances of a computational problem that may reduce the computational complexity of solving that instance or instances. In this paper, we study the value of help bits in the settings of randomized and average-case complexity. Amir, Beigel, and Gasarch (1990) show that for constant $k$, if $k$ instances of a decision problem can be efficiently solved using less than $k$ bits of help, then the problem is in P/poly. We extend this result to the setting of randomized computation: We show that the decision problem is in P/poly if using $\ell$ help bits, $k$ instances of the problem can be efficiently solved with probability greater than $2^{\ell-k}$. The same result holds if using less than $k(1 - h(α))$ help bits (where $h(\cdot)$ is the binary entropy function), we can efficiently solve $(1-α)$ fraction of the instances correctly with non-vanishing probability. We also extend these two results to non-constant but logarithmic $k$. In this case however, instead of showing that the problem is in P/poly we show that it satisfies "$k$-membership comparability," a notion known to be related to solving $k$ instances using less than $k$ bits of help. Next we consider the setting of average-case complexity: Assume that we can solve $k$ instances of a decision problem using some help bits whose entropy is less than $k$ when the $k$ instances are drawn independently from a particular distribution. Then we can efficiently solve an instance drawn from that distribution with probability better than $1/2$. Finally, we show that in the case where $k$ is super-logarithmic, assuming $k$-membership comparability of a decision problem, one cannot prove that the problem is in P/poly by a "black-box proof." △ Less

Submitted 3 August, 2014; originally announced August 2014.

Journal ref: Computational Complexity 2016, pp. 1-27

Receivers for Diffusion-Based Molecular Communication: Exploiting Memory and Sampling Rate

Authors: Reza Mosayebi, Hamidreza Arjmandi, Amin Gohari, Masoumeh Nasiri Kenari, Urbashi Mitra

Abstract: In this paper, a diffusion-based molecular communication channel between two nano-machines is considered. The effect of the amount of memory on performance is characterized, and a simple memory-limited decoder is proposed and its performance is shown to be close to that of the best possible imaginable decoder (without any restriction on the computational complexity or its functional form), using G… ▽ More In this paper, a diffusion-based molecular communication channel between two nano-machines is considered. The effect of the amount of memory on performance is characterized, and a simple memory-limited decoder is proposed and its performance is shown to be close to that of the best possible imaginable decoder (without any restriction on the computational complexity or its functional form), using Genie-aided upper bounds. This effect is specialized for the case of Molecular Concentration Shift Keying; it is shown that a four-bits memory achieved nearly the same performance as infinite memory. Then a general class of threshold decoders is considered and shown not to be optimal for Poisson channel with memory, unless SNR is higher than a value specified in the paper. Another contribution is to show that receiver sampling at a rate higher than the transmission rate, i.e., a multi-read system, can significantly improve the performance. The associated decision rule for this system is shown to be a weighted sum of the samples during each symbol interval. The performance of the system is analyzed using the saddle point approximation. The best performance gains are achieved for an oversampling factor of three. △ Less

Submitted 2 May, 2014; v1 submitted 1 May, 2014; originally announced May 2014.

Comments: Submitted to JSAC

Quantum Achievability Proof via Collision Relative Entropy

Authors: Salman Beigi, Amin Gohari

Abstract: In this paper, we provide a simple framework for deriving one-shot achievable bounds for some problems in quantum information theory. Our framework is based on the joint convexity of the exponential of the collision relative entropy, and is a (partial) quantum generalization of the technique of Yassaee et al. (2013) from classical information theory. Based on this framework, we derive one-shot ach… ▽ More In this paper, we provide a simple framework for deriving one-shot achievable bounds for some problems in quantum information theory. Our framework is based on the joint convexity of the exponential of the collision relative entropy, and is a (partial) quantum generalization of the technique of Yassaee et al. (2013) from classical information theory. Based on this framework, we derive one-shot achievable bounds for the problems of communication over classical-quantum channels, quantum hypothesis testing, and classical data compression with quantum side information. We argue that our one-shot achievable bounds are strong enough to give the asymptotic achievable rates of these problems even up to the second order. △ Less

Submitted 11 January, 2017; v1 submitted 13 December, 2013; originally announced December 2013.

Comments: 12 pages

Journal ref: IEEE Transactions on Information Theory 60(12), 7980-7986 (2014)

Critical Graphs in Index Coding

Authors: Mehrdad Tahmasbi, Amirbehshad Shahrasbi, Amin Gohari

Abstract: In this paper we define critical graphs as minimal graphs that support a given set of rates for the index coding problem, and study them for both the one-shot and asymptotic setups. For the case of equal rates, we find the critical graph with minimum number of edges for both one-shot and asymptotic cases. For the general case of possibly distinct rates, we show that for one-shot and asymptotic lin… ▽ More In this paper we define critical graphs as minimal graphs that support a given set of rates for the index coding problem, and study them for both the one-shot and asymptotic setups. For the case of equal rates, we find the critical graph with minimum number of edges for both one-shot and asymptotic cases. For the general case of possibly distinct rates, we show that for one-shot and asymptotic linear index coding, as well as asymptotic non-linear index coding, each critical graph is a union of disjoint strongly connected subgraphs (USCS). On the other hand, we identify a non-USCS critical graph for a one-shot non-linear index coding problem. Next, we identify a few graph structures that are critical. We also generalize some of our results to the groupcast problem. In addition, we show that the capacity region of the index coding is additive for union of disjoint graphs. △ Less

Submitted 12 April, 2014; v1 submitted 30 November, 2013; originally announced December 2013.

Simulation of a Channel with Another Channel

Authors: Farzin Haddadpour, Mohammad Hossein Yassaee, Salman Beigi, Amin Gohari, Mohammad Reza Aref

Abstract: In this paper, we study the problem of simulating a DMC channel from another DMC channel under an average-case and an exact model. We present several achievability and infeasibility results, with tight characterizations in special cases. In particular for the exact model, we fully characterize when a BSC channel can be simulated from a BEC channel when there is no shared randomness. We also provid… ▽ More In this paper, we study the problem of simulating a DMC channel from another DMC channel under an average-case and an exact model. We present several achievability and infeasibility results, with tight characterizations in special cases. In particular for the exact model, we fully characterize when a BSC channel can be simulated from a BEC channel when there is no shared randomness. We also provide infeasibility and achievability results for simulation of a binary channel from another binary channel in the case of no shared randomness. To do this, we use properties of Rényi capacity of a given order. We also introduce a notion of "channel diameter" which is shown to be additive and satisfy a data processing inequality. △ Less

Submitted 1 December, 2016; v1 submitted 25 May, 2013; originally announced May 2013.

Comments: 31 pages, 10 figures, and some parts of this work were published at ITW 2013

On Maximal Correlation, Hypercontractivity, and the Data Processing Inequality studied by Erkip and Cover

Authors: Venkat Anantharam, Amin Gohari, Sudeep Kamath, Chandra Nair

Abstract: In this paper we provide a new geometric characterization of the Hirschfeld-Gebelein-Rényi maximal correlation of a pair of random $(X,Y)$, as well as of the chordal slope of the nontrivial boundary of the hypercontractivity ribbon of $(X,Y)$ at infinity. The new characterizations lead to simple proofs for some of the known facts about these quantities. We also provide a counterexample to a data p… ▽ More In this paper we provide a new geometric characterization of the Hirschfeld-Gebelein-Rényi maximal correlation of a pair of random $(X,Y)$, as well as of the chordal slope of the nontrivial boundary of the hypercontractivity ribbon of $(X,Y)$ at infinity. The new characterizations lead to simple proofs for some of the known facts about these quantities. We also provide a counterexample to a data processing inequality claimed by Erkip and Cover, and find the correct tight constant for this kind of inequality. △ Less

Submitted 22 April, 2013; originally announced April 2013.

Comments: 11 pages

A Technique for Deriving One-Shot Achievability Results in Network Information Theory

Authors: Mohammad Hossein Yassaee, Mohammad Reza Aref, Amin Gohari

Abstract: This paper proposes a novel technique to prove a one-shot version of achievability results in network information theory. The technique is not based on covering and packing lemmas. In this technique, we use an stochastic encoder and decoder with a particular structure for coding that resembles both the ML and the joint-typicality coders. Although stochastic encoders and decoders do not usually enh… ▽ More This paper proposes a novel technique to prove a one-shot version of achievability results in network information theory. The technique is not based on covering and packing lemmas. In this technique, we use an stochastic encoder and decoder with a particular structure for coding that resembles both the ML and the joint-typicality coders. Although stochastic encoders and decoders do not usually enhance the capacity region, their use simplifies the analysis. The Jensen inequality lies at the heart of error analysis, which enables us to deal with the expectation of many terms coming from stochastic encoders and decoders at once. The technique is illustrated via several examples: point-to-point channel coding, Gelfand-Pinsker, Broadcast channel (Marton), Berger-Tung, Heegard-Berger/Kaspi, Multiple description coding and Joint source-channel coding over a MAC. Most of our one-shot results are new. The asymptotic forms of these expressions is the same as that of classical results. Our one-shot bounds in conjunction with multi-dimensional Berry-Essen CLT imply new results in the finite blocklength regime. In particular applying the one-shot result for the memoryless broadcast channel in the asymptotic case, we get the entire region of Marton's inner bound without any need for time-sharing. △ Less

Submitted 4 March, 2013; originally announced March 2013.

Comments: A short version has been submitted to ISIT 2013

Non-Asymptotic Output Statistics of Random Binning and Its Applications

Authors: Mohammad Hossein Yassaee, Mohammad Reza Aref, Amin Gohari

Abstract: In this paper we develop a finite blocklength version of the Output Statistics of Random Binning (OSRB) framework. The framework is shown to be optimal in the point-to-point case. New second order regions for broadcast channel and wiretap channel with strong secrecy criterion are derived. In this paper we develop a finite blocklength version of the Output Statistics of Random Binning (OSRB) framework. The framework is shown to be optimal in the point-to-point case. New second order regions for broadcast channel and wiretap channel with strong secrecy criterion are derived. △ Less

Submitted 4 March, 2013; originally announced March 2013.

Comments: A shot version has been submitted to ISIT 2013

Diffusion Based Nanonetworking: A New Modulation Technique and Performance Analysis

Authors: Hamidreza Arjmandi, Amin Gohari, Masoume Nasiri Kenari, Farshid Bateni

Abstract: In this letter, we propose a new molecular modulation scheme for nanonetworks. To evaluate the scheme we introduce a more realistic system model for molecule dissemination and propagation processes based on the Poisson distribution. We derive the probability of error of our proposed scheme as well as the previously introduced schemes, including concentration and molecular shift keying modulations… ▽ More In this letter, we propose a new molecular modulation scheme for nanonetworks. To evaluate the scheme we introduce a more realistic system model for molecule dissemination and propagation processes based on the Poisson distribution. We derive the probability of error of our proposed scheme as well as the previously introduced schemes, including concentration and molecular shift keying modulations by taking into account the error propagation effect of previously decoded symbols. Since in our scheme the decoding of the current symbol does not depend on the previously transmitted and decoded symbols, we do not encounter error propagation; and so as our numerical results indicate, the proposed scheme outperforms the previously introduced schemes. We then introduce a general molecular communication system and use information theoretic tools to derive fundamental limits on its probability of error. △ Less

Submitted 25 September, 2012; originally announced September 2012.

On Dimension Bounds for Auxiliary Quantum Systems

Authors: Salman Beigi, Amin Gohari

Abstract: Expressions of several capacity regions in quantum information theory involve an optimization over auxiliary quantum registers. Evaluating such expressions requires bounds on the dimension of the Hilbert space of these auxiliary registers, for which no non-trivial technique is known; we lack a quantum analog of the Carathéodory theorem. In this paper, we develop a new non-Carathéodory-type tool fo… ▽ More Expressions of several capacity regions in quantum information theory involve an optimization over auxiliary quantum registers. Evaluating such expressions requires bounds on the dimension of the Hilbert space of these auxiliary registers, for which no non-trivial technique is known; we lack a quantum analog of the Carathéodory theorem. In this paper, we develop a new non-Carathéodory-type tool for evaluating expressions involving a single quantum auxiliary register and several classical random variables. As we show, such expressions appear in problems of entanglement-assisted Gray-Wyner and entanglement-assisted channel simulation, where the question of whether entanglement helps in these settings is related to that of evaluating expressions with a single quantum auxiliary register. To evaluate such expressions, we argue that develo** a quantum analog of the Carathéodory theorem requires a better understanding of a notion which we call ``quantum conditioning." We then proceed by proving a few results about quantum conditioning, one of which is that quantum conditioning is strictly richer than the usual classical conditioning. △ Less

Submitted 15 November, 2013; v1 submitted 17 July, 2012; originally announced July 2012.

Comments: 30 pages, title changed, structure significantly improved, results unchanged, to appear in IEEE TIT

Journal ref: IEEE Transactions on Information Theory, vol 60, pp. 368-387, January 2014

Secure Channel Simulation

Authors: Amin Gohari, Mohammad Hossein Yassaee, Mohammad Reza Aref

Abstract: In this paper the Output Statistics of Random Binning (OSRB) framework is used to prove a new inner bound for the problem of secure channel simulation. Our results subsume some recent results on the secure function computation. We also provide an achievability result for the problem of simultaneously simulating a channel and creating a shared secret key. A special case of this result generalizes t… ▽ More In this paper the Output Statistics of Random Binning (OSRB) framework is used to prove a new inner bound for the problem of secure channel simulation. Our results subsume some recent results on the secure function computation. We also provide an achievability result for the problem of simultaneously simulating a channel and creating a shared secret key. A special case of this result generalizes the lower bound of Gohari and Anantharam on the source model to include constraints on the rates of the public discussion. △ Less

Submitted 15 July, 2012; originally announced July 2012.

Comments: Short version to appear at ITW 2012. This will be replaced by a longer version soon

Information Theoretic cutting of a cake

Authors: Payam Delgosha, Amin Gohari

Abstract: Cutting a cake is a metaphor for the problem of dividing a resource (cake) among several agents. The problem becomes non-trivial when the agents have different valuations for different parts of the cake (i.e. one agent may like chocolate while the other may like cream). A fair division of the cake is one that takes into account the individual valuations of agents and partitions the cake based on s… ▽ More Cutting a cake is a metaphor for the problem of dividing a resource (cake) among several agents. The problem becomes non-trivial when the agents have different valuations for different parts of the cake (i.e. one agent may like chocolate while the other may like cream). A fair division of the cake is one that takes into account the individual valuations of agents and partitions the cake based on some fairness criterion. Fair division may be accomplished in a distributed or centralized way. Due to its natural and practical appeal, it has been a subject of study in economics. To best of our knowledge the role of partial information in fair division has not been studied so far from an information theoretic perspective. In this paper we study two important algorithms in fair division, namely "divide and choose" and "adjusted winner" for the case of two agents. We quantify the benefit of negotiation in the divide and choose algorithm, and its use in tricking the adjusted winner algorithm. Also we analyze the role of implicit information transmission through actions for the repeated divide and choose problem by finding a trembling hand perfect equilibrium for an specific setup. Lastly we consider a centralized algorithm for maximizing the overall welfare of the agents under the Nash collective utility function (CUF). This corresponds to a clustering problem of the type traditionally studied in data mining and machine learning. Drawing a conceptual link between this problem and the portfolio selection problem in stock markets, we prove an upper bound on the increase of the Nash CUF for a clustering refinement. △ Less

Submitted 24 January, 2016; v1 submitted 18 May, 2012; originally announced May 2012.

Comments: Submitted to IEEE Transactions on Information Theory

Channel simulation via interactive communications

Authors: Mohammad Hossein Yassaee, Amin Gohari, Mohammad Reza Aref

Abstract: In this paper, we study the problem of channel simulation via interactive communication, known as the coordination capacity, in a two-terminal network. We assume that two terminals observe i.i.d.\ copies of two random variables and would like to generate i.i.d.\ copies of two other random variables jointly distributed with the observed random variables. The terminals are provided with two-way comm… ▽ More In this paper, we study the problem of channel simulation via interactive communication, known as the coordination capacity, in a two-terminal network. We assume that two terminals observe i.i.d.\ copies of two random variables and would like to generate i.i.d.\ copies of two other random variables jointly distributed with the observed random variables. The terminals are provided with two-way communication links, and shared common randomness, all at limited rates. Two special cases of this problem are the interactive function computation studied by Ma and Ishwar, and the tradeoff curve between one-way communication and shared randomness studied by Cuff. The latter work had inspired Gohari and Anantharam to study the general problem of channel simulation via interactive communication stated above. However only inner and outer bounds for the special case of no shared randomness were obtained in their work. In this paper we settle this problem by providing an exact computable characterization of the multi-round problem. To show this we employ the technique of "output statistics of random binning" that has been recently developed by the authors. △ Less

Submitted 18 April, 2014; v1 submitted 14 March, 2012; originally announced March 2012.

Comments: Submitted to IEEE Trans. on Information Theory

Coordination via a relay

Authors: Farzin Haddadpour, Mohammad Hossein Yassaee, Amin Gohari, Mohammad Reza Aref

Abstract: In this paper, we study the problem of coordinating two nodes which can only exchange information via a relay at limited rates. The nodes are allowed to do a two-round interactive two-way communication with the relay, after which they should be able to generate i.i.d. copies of two random variables with a given joint distribution within a vanishing total variation distance. We prove inner and oute… ▽ More In this paper, we study the problem of coordinating two nodes which can only exchange information via a relay at limited rates. The nodes are allowed to do a two-round interactive two-way communication with the relay, after which they should be able to generate i.i.d. copies of two random variables with a given joint distribution within a vanishing total variation distance. We prove inner and outer bounds on the coordination capacity region for this problem. Our inner bound is proved using the technique of "output statistics of random binning" that has recently been developed by Yassaee, et al. △ Less

Submitted 4 March, 2012; originally announced March 2012.

Comments: Submitted to ISIT 2012

Achievability proof via output statistics of random binning

Authors: Mohammad Hossein Yassaee, Mohammad Reza Aref, Amin Gohari

Abstract: This paper introduces a new and ubiquitous framework for establishing achievability results in \emph{network information theory} (NIT) problems. The framework uses random binning arguments and is based on a duality between channel and source coding problems. {Further,} the framework uses pmf approximation arguments instead of counting and typicality. This allows for proving coordination and \emph{… ▽ More This paper introduces a new and ubiquitous framework for establishing achievability results in \emph{network information theory} (NIT) problems. The framework uses random binning arguments and is based on a duality between channel and source coding problems. {Further,} the framework uses pmf approximation arguments instead of counting and typicality. This allows for proving coordination and \emph{strong} secrecy problems where certain statistical conditions on the distribution of random variables need to be satisfied. These statistical conditions include independence between messages and eavesdropper's observations in secrecy problems and closeness to a certain distribution (usually, i.i.d. distribution) in coordination problems. One important feature of the framework is to enable one {to} add an eavesdropper and obtain a result on the secrecy rates "for free." We make a case for generality of the framework by studying examples in the variety of settings containing channel coding, lossy source coding, joint source-channel coding, coordination, strong secrecy, feedback and relaying. In particular, by investigating the framework for the lossy source coding problem over broadcast channel, it is shown that the new framework provides a simple alternative scheme to \emph{hybrid} coding scheme. Also, new results on secrecy rate region (under strong secrecy criterion) of wiretap broadcast channel and wiretap relay channel are derived. In a set of accompanied papers, we have shown the usefulness of the framework to establish achievability results for coordination problems including interactive channel simulation, coordination via relay and channel simulation via another channel. △ Less

Submitted 21 August, 2014; v1 submitted 4 March, 2012; originally announced March 2012.

Comments: IEEE Transactions on Information Theory, accepted

On Marton's inner bound for broadcast channels

Authors: Amin Gohari, Chandra Nair, Venkat Anantharam

Abstract: Marton's inner bound is the best known achievable region for a general discrete memoryless broadcast channel. To compute Marton's inner bound one has to solve an optimization problem over a set of joint distributions on the input and auxiliary random variables. The optimizers turn out to be structured in many cases. Finding properties of optimizers not only results in efficient evaluation of the r… ▽ More Marton's inner bound is the best known achievable region for a general discrete memoryless broadcast channel. To compute Marton's inner bound one has to solve an optimization problem over a set of joint distributions on the input and auxiliary random variables. The optimizers turn out to be structured in many cases. Finding properties of optimizers not only results in efficient evaluation of the region, but it may also help one to prove factorization of Marton's inner bound (and thus its optimality). The first part of this paper formulates this factorization approach explicitly and states some conjectures and results along this line. The second part of this paper focuses primarily on the structure of the optimizers. This section is inspired by a new binary inequality that recently resulted in a very simple characterization of the sum-rate of Marton's inner bound for binary input broadcast channels. This prompted us to investigate whether this inequality can be extended to larger cardinality input alphabets. We show that several of the results for the binary input case do carry over for higher cardinality alphabets and we present a collection of results that help restrict the search space of probability distributions to evaluate the boundary of Marton's inner bound in the general case. We also prove a new inequality for the binary skew-symmetric broadcast channel that yields a very simple characterization of the entire Marton inner bound for this channel. △ Less

Submitted 4 February, 2012; originally announced February 2012.

Comments: Submitted to ISIT 2012