Showing 1–10 of 10 results for author: Kaced, T

The Entropy Region is not Closed Under Duality

Authors: Tarik Kaced

Abstract: We import a duality notion coming from polymatroids to define duality for information inequalities. We show that the entropy region for $n\ge 5$ is not closed under duality. Our result answers an open question of Matùš (1992). We import a duality notion coming from polymatroids to define duality for information inequalities. We show that the entropy region for $n\ge 5$ is not closed under duality. Our result answers an open question of Matùš (1992). △ Less

Submitted 13 November, 2016; originally announced November 2016.

Comments: submitted

Info-Clustering: A Mathematical Theory for Data Clustering

Authors: Chung Chan, Ali Al-Bashabsheh, Qiaoqiao Zhou, Tarik Kaced, Tie Liu

Abstract: We formulate an info-clustering paradigm based on a multivariate information measure, called multivariate mutual information, that naturally extends Shannon's mutual information between two random variables to the multivariate case involving more than two random variables. With proper model reductions, we show that the paradigm can be applied to study the human genome and connectome in a more mean… ▽ More We formulate an info-clustering paradigm based on a multivariate information measure, called multivariate mutual information, that naturally extends Shannon's mutual information between two random variables to the multivariate case involving more than two random variables. With proper model reductions, we show that the paradigm can be applied to study the human genome and connectome in a more meaningful way than the conventional algorithmic approach. Not only can info-clustering provide justifications and refinements to some existing techniques, but it also inspires new computationally feasible solutions. △ Less

Submitted 11 December, 2016; v1 submitted 4 May, 2016; originally announced May 2016.

Comments: In celebration of Claude Shannon's Centenary

Conditional Information Inequalities and Combinatorial Applications

Authors: Tarik Kaced, Andrei Romashchenko, Nikolay Vereshchagin

Abstract: We show that the inequality $H(A \mid B,X) + H(A \mid B,Y) \le H(A\mid B)$ for jointly distributed random variables $A,B,X,Y$, which does not hold in general case, holds under some natural condition on the support of the probability distribution of $A,B,X,Y$. This result generalizes a version of the conditional Ingleton inequality: if for some distribution $I(X: Y \mid A) = H(A\mid X,Y)=0$, then… ▽ More We show that the inequality $H(A \mid B,X) + H(A \mid B,Y) \le H(A\mid B)$ for jointly distributed random variables $A,B,X,Y$, which does not hold in general case, holds under some natural condition on the support of the probability distribution of $A,B,X,Y$. This result generalizes a version of the conditional Ingleton inequality: if for some distribution $I(X: Y \mid A) = H(A\mid X,Y)=0$, then $I(A : B) \le I(A : B \mid X) + I(A: B \mid Y) + I(X : Y)$. We present two applications of our result. The first one is the following easy-to-formulate combinatorial theorem: assume that the edges of a bipartite graph are partitioned into $K$ matchings such that for each pair (left vertex $x$, right vertex $y$) there is at most one matching in the partition involving both $x$ and $y$; assume further that the degree of each left vertex is at least $L$ and the degree of each right vertex is at least $R$. Then $K\ge LR$. The second application is a new method to prove lower bounds for biclique coverings of bipartite graphs. △ Less

Submitted 13 September, 2017; v1 submitted 20 January, 2015; originally announced January 2015.

Group Testing with Prior Statistics

Authors: Tongxin Li, Chun Lam Chan, Wenhao Huang, Tarik Kaced, Sidharth Jaggi

Abstract: We consider a new group testing model wherein each item is a binary random variable defined by an a priori probability of being defective. We assume that each probability is small and that items are independent, but not necessarily identically distributed. The goal of group testing algorithms is to identify with high probability the subset of defectives via non-linear (disjunctive) binary measurem… ▽ More We consider a new group testing model wherein each item is a binary random variable defined by an a priori probability of being defective. We assume that each probability is small and that items are independent, but not necessarily identically distributed. The goal of group testing algorithms is to identify with high probability the subset of defectives via non-linear (disjunctive) binary measurements. Our main contributions are two classes of algorithms: (1) adaptive algorithms with tests based either on a maximum entropy principle, or on a Shannon-Fano/Huffman code; (2) non-adaptive algorithms. Under loose assumptions and with high probability, our algorithms only need a number of measurements that is close to the information-theoretic lower bound, up to an explicitly-calculated universal constant factor. We provide simulations to support our results. △ Less

Submitted 21 July, 2018; v1 submitted 15 January, 2014; originally announced January 2014.

Comments: 23 pages, 12 figures, presented in ISIT 2014, Honolulu

Journal ref: Information Theory (ISIT), 2014 IEEE International Symposium on, 2346-2350

Equivalence of Two Proof Techniques for Non-Shannon-type Inequalities

Authors: Tarik Kaced

Abstract: We compare two different techniques for proving non-Shannon-type information inequalities. The first one is the original Zhang-Yeung's method, commonly referred to as the copy/pasting lemma/trick. The copy lemma was used to derive the first conditional and unconditional non-Shannon-type inequalities. The second technique first appeared in Makarychev et al paper [7] and is based on a coding lemma f… ▽ More We compare two different techniques for proving non-Shannon-type information inequalities. The first one is the original Zhang-Yeung's method, commonly referred to as the copy/pasting lemma/trick. The copy lemma was used to derive the first conditional and unconditional non-Shannon-type inequalities. The second technique first appeared in Makarychev et al paper [7] and is based on a coding lemma from Ahlswede and Körner works. We first emphasize the importance of balanced inequalities and provide a simpler proof of a theorem of Chan's for the case of Shannon-type inequalities. We compare the power of various proof systems based on a single technique. △ Less

Submitted 13 February, 2013; originally announced February 2013.

Comments: 5 pages paper submitted to ISIT 2013

Conditional Information Inequalities for Entropic and Almost Entropic Points

Authors: Tarik Kaced, Andrei Romashchenko

Abstract: We study conditional linear information inequalities, i.e., linear inequalities for Shannon entropy that hold for distributions whose entropies meet some linear constraints. We prove that some conditional information inequalities cannot be extended to any unconditional linear inequalities. Some of these conditional inequalities hold for almost entropic points, while others do not. We also discuss… ▽ More We study conditional linear information inequalities, i.e., linear inequalities for Shannon entropy that hold for distributions whose entropies meet some linear constraints. We prove that some conditional information inequalities cannot be extended to any unconditional linear inequalities. Some of these conditional inequalities hold for almost entropic points, while others do not. We also discuss some counterparts of conditional information inequalities for Kolmogorov complexity. △ Less

Submitted 16 August, 2013; v1 submitted 24 July, 2012; originally announced July 2012.

Comments: Submitted to the IEEE Transactions on Information Theory

Journal ref: IEEE Transactions on Information Theory 59(11), 2013, pp. 7149-7167

On the Non-robustness of Essentially Conditional Information Inequalities

Authors: Tarik Kaced, Andrei Romashchenko

Abstract: We show that two essentially conditional linear inequalities for Shannon's entropies (including the Zhang-Yeung'97 conditional inequality) do not hold for asymptotically entropic points. This means that these inequalities are non-robust in a very strong sense. This result raises the question of the meaning of these inequalities and the validity of their use in practice-oriented applications. We show that two essentially conditional linear inequalities for Shannon's entropies (including the Zhang-Yeung'97 conditional inequality) do not hold for asymptotically entropic points. This means that these inequalities are non-robust in a very strong sense. This result raises the question of the meaning of these inequalities and the validity of their use in practice-oriented applications. △ Less

Submitted 23 July, 2012; originally announced July 2012.

Comments: ITW 2012 Conference paper, 5 pages, 1 figure

Conditional and unconditional information inequalities: an algebraic example

Authors: Tarik Kaced, Andrei Romashchenko

Abstract: We provide a simple example showing that some conditional information inequalities (even in a weak form) cannot be derived from unconditional inequalities. We provide a simple example showing that some conditional information inequalities (even in a weak form) cannot be derived from unconditional inequalities. △ Less

Submitted 30 January, 2012; originally announced January 2012.

Comments: Short note

On essentially conditional information inequalities

Authors: Tarik Kaced, Andrei Romashchenko

Abstract: In 1997, Z.Zhang and R.W.Yeung found the first example of a conditional information inequality in four variables that is not "Shannon-type". This linear inequality for entropies is called conditional (or constraint) since it holds only under condition that some linear equations are satisfied for the involved entropies. Later, the same authors and other researchers discovered several unconditional… ▽ More In 1997, Z.Zhang and R.W.Yeung found the first example of a conditional information inequality in four variables that is not "Shannon-type". This linear inequality for entropies is called conditional (or constraint) since it holds only under condition that some linear equations are satisfied for the involved entropies. Later, the same authors and other researchers discovered several unconditional information inequalities that do not follow from Shannon's inequalities for entropy. In this paper we show that some non Shannon-type conditional inequalities are "essentially" conditional, i.e., they cannot be extended to any unconditional inequality. We prove one new essentially conditional information inequality for Shannon's entropy and discuss conditional information inequalities for Kolmogorov complexity. △ Less

Submitted 26 September, 2011; v1 submitted 13 March, 2011; originally announced March 2011.

Comments: v4: substantial corrections; 13 pages

Almost-perfect secret sharing

Authors: Tarik Kaced

Abstract: Splitting a secret s between several participants, we generate (for each value of s) shares for all participants. The goal: authorized groups of participants should be able to reconstruct the secret but forbidden ones get no information about it. In this paper we introduce several notions of non- perfect secret sharing, where some small information leak is permitted. We study its relation to the K… ▽ More Splitting a secret s between several participants, we generate (for each value of s) shares for all participants. The goal: authorized groups of participants should be able to reconstruct the secret but forbidden ones get no information about it. In this paper we introduce several notions of non- perfect secret sharing, where some small information leak is permitted. We study its relation to the Kolmogorov complexity version of secret sharing (establishing some connection in both directions) and the effects of changing the secret size (showing that we can decrease the size of the secret and the information leak at the same time). △ Less

Submitted 17 November, 2011; v1 submitted 13 March, 2011; originally announced March 2011.

Comments: Acknowledgments added