-
Stochastic proof of the sharp symmetrized Talagrand inequality
Authors:
Thomas A. Courtade,
Max Fathi,
Dan Mikulincer
Abstract:
We give a new proof of the sharp symmetrized form of Talagrand's transport-entropy inequality. Compared to stochastic proofs of other Gaussian functional inequalities, the new idea here is a certain coupling induced by time-reversed martingale representations.
We give a new proof of the sharp symmetrized form of Talagrand's transport-entropy inequality. Compared to stochastic proofs of other Gaussian functional inequalities, the new idea here is a certain coupling induced by time-reversed martingale representations.
△ Less
Submitted 12 July, 2024;
originally announced July 2024.
-
Stability of the Poincaré-Korn inequality
Authors:
Thomas A. Courtade,
Max Fathi
Abstract:
We resolve a question of Carrapatoso et al. on Gaussian optimality for the sharp constant in Poincaré-Korn inequalities, under a moment constraint. We also prove stability, showing that measures with near-optimal constant are quantitatively close to standard Gaussian.
We resolve a question of Carrapatoso et al. on Gaussian optimality for the sharp constant in Poincaré-Korn inequalities, under a moment constraint. We also prove stability, showing that measures with near-optimal constant are quantitatively close to standard Gaussian.
△ Less
Submitted 2 May, 2024;
originally announced May 2024.
-
Stability of Klartag's improved Lichnerowicz inequality
Authors:
Thomas A. Courtade,
Max Fathi
Abstract:
In a recent work, Klartag gave an improved version of Lichnerowicz' spectral gap bound for uniformly log-concave measures, which improves on the classical estimate by taking into account the covariance matrix. We analyze the equality cases in Klartag's bound, showing that it can be further improved whenever the measure has no Gaussian factor. Additionally, we give a quantitative improvement for lo…
▽ More
In a recent work, Klartag gave an improved version of Lichnerowicz' spectral gap bound for uniformly log-concave measures, which improves on the classical estimate by taking into account the covariance matrix. We analyze the equality cases in Klartag's bound, showing that it can be further improved whenever the measure has no Gaussian factor. Additionally, we give a quantitative improvement for log-concave measures with finite Fisher information.
△ Less
Submitted 18 April, 2024;
originally announced April 2024.
-
Rigid characterizations of probability measures through independence, with applications
Authors:
Thomas A. Courtade
Abstract:
Three equivalent characterizations of probability measures through independence criteria are given. These characterizations lead to a family of Brascamp--Lieb-type inequalities for relative entropy, determine equilibrium states and sharp rates of convergence for certain linear Boltzmann-type dynamics, and unify an assortment of $L^2$ inequalities in probability.
Three equivalent characterizations of probability measures through independence criteria are given. These characterizations lead to a family of Brascamp--Lieb-type inequalities for relative entropy, determine equilibrium states and sharp rates of convergence for certain linear Boltzmann-type dynamics, and unify an assortment of $L^2$ inequalities in probability.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
HWI inequalities in discrete spaces via couplings
Authors:
Thomas A. Courtade,
Max Fathi
Abstract:
HWI inequalities are interpolation inequalities relating entropy, Fisher information and optimal transport distances. We adapt an argument of Y. Wu for proving the Gaussian HWI inequality via a coupling argument to the discrete setting, establishing new interpolation inequalities for the discrete hypercube and the discrete torus. In particular, we obtain an improvement of the modified logarithmic…
▽ More
HWI inequalities are interpolation inequalities relating entropy, Fisher information and optimal transport distances. We adapt an argument of Y. Wu for proving the Gaussian HWI inequality via a coupling argument to the discrete setting, establishing new interpolation inequalities for the discrete hypercube and the discrete torus. In particular, we obtain an improvement of the modified logarithmic Sobolev inequality for the discrete hypercube of Bobkov and Tetali.
△ Less
Submitted 3 December, 2023;
originally announced December 2023.
-
Mean Estimation Under Heterogeneous Privacy Demands
Authors:
Syomantak Chaudhuri,
Konstantin Miagkov,
Thomas A. Courtade
Abstract:
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation, where each user can impose their own distinc…
▽ More
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation, where each user can impose their own distinct privacy level. The algorithm we propose is shown to be minimax optimal and has a near-linear run-time. Our results elicit an interesting saturation phenomenon that occurs. Namely, the privacy requirements of the most stringent users dictate the overall error rates. As a consequence, users with less but differing privacy requirements are all given more privacy than they require, in equal amounts. In other words, these privacy-indifferent users are given a nontrivial degree of privacy for free, without any sacrifice in the performance of the estimator.
△ Less
Submitted 19 October, 2023;
originally announced October 2023.
-
Mean Estimation Under Heterogeneous Privacy: Some Privacy Can Be Free
Authors:
Syomantak Chaudhuri,
Thomas A. Courtade
Abstract:
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional DP formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation under heterogeneous DP constraints, where…
▽ More
Differential Privacy (DP) is a well-established framework to quantify privacy loss incurred by any algorithm. Traditional DP formulations impose a uniform privacy requirement for all users, which is often inconsistent with real-world scenarios in which users dictate their privacy preferences individually. This work considers the problem of mean estimation under heterogeneous DP constraints, where each user can impose their own distinct privacy level. The algorithm we propose is shown to be minimax optimal when there are two groups of users with distinct privacy levels. Our results elicit an interesting saturation phenomenon that occurs as one group's privacy level is relaxed, while the other group's privacy level remains constant. Namely, after a certain point, further relaxing the privacy requirement of the former group does not improve the performance of the minimax optimal mean estimator. Thus, the central server can offer a certain degree of privacy without any sacrifice in performance.
△ Less
Submitted 27 April, 2023;
originally announced May 2023.
-
Entropy Inequalities and Gaussian Comparisons
Authors:
Efe Aras,
Thomas A. Courtade
Abstract:
We establish a general class of entropy inequalities that take the concise form of Gaussian comparisons. The main result unifies many classical and recent results, including the Shannon-Stam inequality, the Brunn-Minkowski inequality, the Zamir-Feder inequality, the Brascamp-Lieb and Barthe inequalities, the Anantharam-Jog-Nair inequality, and others.
We establish a general class of entropy inequalities that take the concise form of Gaussian comparisons. The main result unifies many classical and recent results, including the Shannon-Stam inequality, the Brunn-Minkowski inequality, the Zamir-Feder inequality, the Brascamp-Lieb and Barthe inequalities, the Anantharam-Jog-Nair inequality, and others.
△ Less
Submitted 28 June, 2022;
originally announced June 2022.
-
Equality cases in the Anantharam-Jog-Nair inequality
Authors:
Efe Aras,
Thomas A. Courtade,
Albert Zhang
Abstract:
Anantharam, Jog and Nair recently unified the Shannon-Stam inequality and the entropic form of the Brascamp-Lieb inequalities under a common inequality. They left open the problems of extremizability and characterization of extremizers. Both questions are resolved in the present paper.
Anantharam, Jog and Nair recently unified the Shannon-Stam inequality and the entropic form of the Brascamp-Lieb inequalities under a common inequality. They left open the problems of extremizability and characterization of extremizers. Both questions are resolved in the present paper.
△ Less
Submitted 23 June, 2022;
originally announced June 2022.
-
Linear Models are Most Favorable among Generalized Linear Models
Authors:
Kuan-Yun Lee,
Thomas A. Courtade
Abstract:
We establish a nonasymptotic lower bound on the $L_2$ minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof…
▽ More
We establish a nonasymptotic lower bound on the $L_2$ minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cramér-Rao bound for log-concave priors, established by Aras et al. (2019).
△ Less
Submitted 9 June, 2020;
originally announced June 2020.
-
Euclidean Forward-Reverse Brascamp-Lieb Inequalities: Finiteness, Structure and Extremals
Authors:
Thomas A. Courtade,
**gbo Liu
Abstract:
A new proof is given for the fact that centered gaussian functions saturate the Euclidean forward-reverse Brascamp-Lieb inequalities, extending the Brascamp-Lieb and Barthe theorems. A duality principle for best constants is also developed, which generalizes the fact that the best constants in the Brascamp-Lieb and Barthe inequalities are equal. Finally, as the title hints, the main results concer…
▽ More
A new proof is given for the fact that centered gaussian functions saturate the Euclidean forward-reverse Brascamp-Lieb inequalities, extending the Brascamp-Lieb and Barthe theorems. A duality principle for best constants is also developed, which generalizes the fact that the best constants in the Brascamp-Lieb and Barthe inequalities are equal. Finally, as the title hints, the main results concerning finiteness, structure and gaussian-extremizability for the Brascamp-Lieb inequality due to Bennett, Carbery, Christ and Tao are generalized to the setting of the forward-reverse Brascamp-Lieb inequality.
△ Less
Submitted 29 August, 2019; v1 submitted 29 July, 2019;
originally announced July 2019.
-
A Family of Bayesian Cramér-Rao Bounds, and Consequences for Log-Concave Priors
Authors:
Efe Aras,
Kuan-Yun Lee,
Ashwin Pananjady,
Thomas A. Courtade
Abstract:
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cramér-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the known Bayesian Cramér-Rao bound (or van Trees inequality), and its less widely known entropic improvement due to Efroimovich. For the setting of a log-concave prior,…
▽ More
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cramér-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the known Bayesian Cramér-Rao bound (or van Trees inequality), and its less widely known entropic improvement due to Efroimovich. For the setting of a log-concave prior, we obtain a Bayesian Cramér-Rao bound which holds for any (possibly biased) estimator and, unlike the van Trees inequality, does not depend on the Fisher information of the prior.
△ Less
Submitted 22 February, 2019;
originally announced February 2019.
-
Transportation Proof of an inequality by Anantharam, Jog and Nair
Authors:
Thomas A. Courtade
Abstract:
Anantharam, Jog and Nair recently put forth an entropic inequality which simultaneously generalizes the Shannon-Stam entropy power inequality and the Brascamp-Lieb inequality in entropic form. We give a brief proof of their result based on optimal transport.
Anantharam, Jog and Nair recently put forth an entropic inequality which simultaneously generalizes the Shannon-Stam entropy power inequality and the Brascamp-Lieb inequality in entropic form. We give a brief proof of their result based on optimal transport.
△ Less
Submitted 31 January, 2019; v1 submitted 30 January, 2019;
originally announced January 2019.
-
Stability of the Bakry-Emery theorem on $\mathbb{R}^n$
Authors:
Thomas A. Courtade,
Max Fathi
Abstract:
We prove stability estimates for the Bakry-Emery bound on Poincaré and logarithmic Sobolev constants of uniformly log-concave measures. In particular, we improve the quantitative bound in a result of De Philippis and Figalli asserting that if a $1$-uniformly log-concave measure has almost the same Poincaré constant as the standard Gaussian measure, then it almost splits off a Gaussian factor, and…
▽ More
We prove stability estimates for the Bakry-Emery bound on Poincaré and logarithmic Sobolev constants of uniformly log-concave measures. In particular, we improve the quantitative bound in a result of De Philippis and Figalli asserting that if a $1$-uniformly log-concave measure has almost the same Poincaré constant as the standard Gaussian measure, then it almost splits off a Gaussian factor, and establish similar new results for logarithmic Sobolev inequalities. As a consequence, we obtain dimension-free stability estimates for Gaussian concentration of Lipschitz functions. The proofs are based on Stein's method, optimal transport, and an approximate integration by parts identity relating measures and approximate optimizers in the associated functional inequality.
△ Less
Submitted 14 September, 2018; v1 submitted 25 July, 2018;
originally announced July 2018.
-
Bounds on the Poincaré constant for convolution measures
Authors:
Thomas A. Courtade
Abstract:
We establish a Shearer-type inequality for the Poincaré constant, showing that the Poincaré constant corresponding to the convolution of a collection of measures can be nontrivially controlled by the Poincaré constants corresponding to convolutions of subsets of measures. This implies, for example, that the Poincaré constant is non-increasing along the central limit theorem. We also establish a di…
▽ More
We establish a Shearer-type inequality for the Poincaré constant, showing that the Poincaré constant corresponding to the convolution of a collection of measures can be nontrivially controlled by the Poincaré constants corresponding to convolutions of subsets of measures. This implies, for example, that the Poincaré constant is non-increasing along the central limit theorem. We also establish a dimension-free stability estimate for subadditivity of the Poincaré constant on convolutions which uniformly improves an earlier one-dimensional estimate of a similar nature by Johnson (2004). As a byproduct of our arguments, we find that the monotone properties of entropy, Fisher information and the Poincaré constant along the CLT find a common root in Shearer's inequality.
△ Less
Submitted 29 June, 2018;
originally announced July 2018.
-
Worst-case vs Average-case Design for Estimation from Fixed Pairwise Comparisons
Authors:
Ashwin Pananjady,
Cheng Mao,
Vidya Muthukumar,
Martin J. Wainwright,
Thomas A. Courtade
Abstract:
Pairwise comparison data arises in many domains, including tournament rankings, web search, and preference elicitation. Given noisy comparisons of a fixed subset of pairs of items, we study the problem of estimating the underlying comparison probabilities under the assumption of strong stochastic transitivity (SST). We also consider the noisy sorting subclass of the SST model. We show that when th…
▽ More
Pairwise comparison data arises in many domains, including tournament rankings, web search, and preference elicitation. Given noisy comparisons of a fixed subset of pairs of items, we study the problem of estimating the underlying comparison probabilities under the assumption of strong stochastic transitivity (SST). We also consider the noisy sorting subclass of the SST model. We show that when the assignment of items to the topology is arbitrary, these permutation-based models, unlike their parametric counterparts, do not admit consistent estimation for most comparison topologies used in practice. We then demonstrate that consistent estimation is possible when the assignment of items to the topology is randomized, thus establishing a dichotomy between worst-case and average-case designs. We propose two estimators in the average-case setting and analyze their risk, showing that it depends on the comparison topology only through the degree sequence of the topology. The rates achieved by these estimators are shown to be optimal for a large class of graphs. Our results are corroborated by simulations on multiple comparison topologies.
△ Less
Submitted 19 July, 2017;
originally announced July 2017.
-
Denoising Linear Models with Permuted Data
Authors:
Ashwin Pananjady,
Martin J. Wainwright,
Thomas A. Courtade
Abstract:
The multivariate linear regression model with shuffled data and additive Gaussian noise arises in various correspondence estimation and matching problems. Focusing on the denoising aspect of this problem, we provide a characterization the minimax error rate that is sharp up to logarithmic factors. We also analyze the performance of two versions of a computationally efficient estimator, and establi…
▽ More
The multivariate linear regression model with shuffled data and additive Gaussian noise arises in various correspondence estimation and matching problems. Focusing on the denoising aspect of this problem, we provide a characterization the minimax error rate that is sharp up to logarithmic factors. We also analyze the performance of two versions of a computationally efficient estimator, and establish their consistency for a large range of input parameters. Finally, we provide an exact algorithm for the noiseless problem and demonstrate its performance on an image point-cloud matching task. Our analysis also extends to datasets with outliers.
△ Less
Submitted 24 April, 2017;
originally announced April 2017.
-
A Counterexample to the Vector Generalization of Costa's EPI, and Partial Resolution
Authors:
Thomas A. Courtade,
Guangyue Han,
Yaochen Wu
Abstract:
We give a counterexample to the vector generalization of Costa's entropy power inequality (EPI) due to Liu, Liu, Poor and Shamai. In particular, the claimed inequality can fail if the matix-valued parameter in the convex combination does not commute with the covariance of the additive Gaussian noise. Conversely, the inequality holds if these two matrices commute.
We give a counterexample to the vector generalization of Costa's entropy power inequality (EPI) due to Liu, Liu, Poor and Shamai. In particular, the claimed inequality can fail if the matix-valued parameter in the convex combination does not commute with the covariance of the additive Gaussian noise. Conversely, the inequality holds if these two matrices commute.
△ Less
Submitted 20 April, 2017;
originally announced April 2017.
-
Existence of Stein Kernels under a Spectral Gap, and Discrepancy Bound
Authors:
Thomas A. Courtade,
Max Fathi,
Ashwin Pananjady
Abstract:
We establish existence of Stein kernels for probability measures on $\mathbb{R}^d$ satisfying a Poincaré inequality, and obtain bounds on the Stein discrepancy of such measures. Applications to quantitative central limit theorems are discussed, including a new CLT in Wasserstein distance $W_2$ with optimal rate and dependence on the dimension. As a byproduct, we obtain a stability version of an es…
▽ More
We establish existence of Stein kernels for probability measures on $\mathbb{R}^d$ satisfying a Poincaré inequality, and obtain bounds on the Stein discrepancy of such measures. Applications to quantitative central limit theorems are discussed, including a new CLT in Wasserstein distance $W_2$ with optimal rate and dependence on the dimension. As a byproduct, we obtain a stability version of an estimate of the Poincaré constant of probability measures under a second moment constraint. The results extend more generally to the setting of converse weighted Poincaré inequalities. The proof is based on simple arguments of calculus of variations.
Further, we establish two general properties enjoyed by the Stein discrepancy, holding whenever a Stein kernel exists: Stein discrepancy is strictly decreasing along the CLT, and it controls the skewness of a random vector.
△ Less
Submitted 8 March, 2018; v1 submitted 22 March, 2017;
originally announced March 2017.
-
Information-Theoretic Perspectives on Brascamp-Lieb Inequality and Its Reverse
Authors:
**gbo Liu,
Thomas A. Courtade,
Paul Cuff,
Sergio Verdu
Abstract:
We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified approach to functional inequalities such as the variational formula of Rényi entropy, hypercontractivity and its reverse, strong data processing inequalities, a…
▽ More
We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified approach to functional inequalities such as the variational formula of Rényi entropy, hypercontractivity and its reverse, strong data processing inequalities, and transportation-cost inequalities, whose utility in the proofs of various coding theorems has gained growing popularity recently. We show that our information-theoretic setting is convenient for proving properties such as data processing, tensorization, convexity (Riesz-Thorin interpolation) and Gaussian optimality. In particular, we elaborate on a "doubling trick" used by Lieb and Geng-Nair to prove several results on Gaussian optimality. Several applications are discussed, including a generalization of the Brascamp-Lieb inequality involving Gaussian random transformations, the determination of Wyner's common information of vector Gaussian sources, and the achievable rate region of certain key generation problems in the case of vector Gaussian sources.
△ Less
Submitted 3 December, 2017; v1 submitted 20 February, 2017;
originally announced February 2017.
-
Wasserstein Stability of the Entropy Power Inequality for Log-Concave Densities
Authors:
Thomas A. Courtade,
Max Fathi,
Ashwin Pananjady
Abstract:
We establish quantitative stability results for the entropy power inequality (EPI). Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic Wasserstein distance. Further, if one of the densities is log-concave and the other is Gaussian, then the deficit in the EPI can be controlled in terms of the $L^1$-W…
▽ More
We establish quantitative stability results for the entropy power inequality (EPI). Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic Wasserstein distance. Further, if one of the densities is log-concave and the other is Gaussian, then the deficit in the EPI can be controlled in terms of the $L^1$-Wasserstein distance. As a counterpoint, an example shows that the EPI can be unstable with respect to the quadratic Wasserstein distance when densities are uniformly log-concave on sets of measure arbitrarily close to one. Our stability results can be extended to non-log-concave densities, provided certain regularity conditions are met. The proofs are based on optimal transportation.
△ Less
Submitted 25 October, 2016;
originally announced October 2016.
-
Novel probabilistic models of spatial genetic ancestry with applications to stratification correction in genome-wide association studies
Authors:
Anand Bhaskar,
Adel Javanmard,
Thomas A. Courtade,
David Tse
Abstract:
Genetic variation in human populations is influenced by geographic ancestry due to spatial locality in historical mating and migration patterns. Spatial population structure in genetic datasets has been traditionally analyzed using either model-free algorithms, such as principal components analysis (PCA) and multidimensional scaling, or using explicit spatial probabilistic models of allele frequen…
▽ More
Genetic variation in human populations is influenced by geographic ancestry due to spatial locality in historical mating and migration patterns. Spatial population structure in genetic datasets has been traditionally analyzed using either model-free algorithms, such as principal components analysis (PCA) and multidimensional scaling, or using explicit spatial probabilistic models of allele frequency evolution. We develop a general probabilistic model and an associated inference algorithm that unify the model-based and data-driven approaches to visualizing and inferring population structure. Our algorithm, Geographic Ancestry Positioning (GAP), relates local genetic distances between samples to their spatial distances, and can be used for visually discerning population structure as well as accurately inferring the spatial origin of individuals on a two-dimensional continuum. On both simulated and several real datasets from diverse human populations, GAP exhibits substantially lower error in reconstructing spatial ancestry coordinates compared to PCA.
Our spatial inference algorithm can also be effectively applied to the problem of population stratification in genome-wide association studies (GWAS), where hidden population structure can create fictitious associations when population ancestry is correlated with both the genotype and the trait. We develop an association test that uses the ancestry coordinates inferred by GAP to accurately account for ancestry-induced correlations in GWAS. Based on simulations and analysis of a dataset of 10 metabolic traits measured in a Northern Finland cohort, which is known to exhibit significant population structure, we find that our method has superior power to current approaches.
△ Less
Submitted 25 October, 2016; v1 submitted 24 October, 2016;
originally announced October 2016.
-
Monotonicity of Entropy and Fisher Information: A Quick Proof via Maximal Correlation
Authors:
Thomas A. Courtade
Abstract:
A simple proof is given for the monotonicity of entropy and Fisher information associated to sums of i.i.d. random variables. The proof relies on a characterization of maximal correlation for partial sums due to Dembo, Kagan and Shepp.
A simple proof is given for the monotonicity of entropy and Fisher information associated to sums of i.i.d. random variables. The proof relies on a characterization of maximal correlation for partial sums due to Dembo, Kagan and Shepp.
△ Less
Submitted 13 October, 2016;
originally announced October 2016.
-
Links between the Logarithmic Sobolev Inequality and the convolution inequalities for Entropy and Fisher Information
Authors:
Thomas A. Courtade
Abstract:
Relative to the Gaussian measure on $\mathbb{R}^d$, entropy and Fisher information are famously related via Gross' logarithmic Sobolev inequality (LSI). These same functionals also separately satisfy convolution inequalities, as proved by Stam. We establish a dimension-free inequality that interpolates among these relations. Several interesting corollaries follow: (i) the deficit in the LSI satisf…
▽ More
Relative to the Gaussian measure on $\mathbb{R}^d$, entropy and Fisher information are famously related via Gross' logarithmic Sobolev inequality (LSI). These same functionals also separately satisfy convolution inequalities, as proved by Stam. We establish a dimension-free inequality that interpolates among these relations. Several interesting corollaries follow: (i) the deficit in the LSI satisfies a convolution inequality itself; (ii) the deficit in the LSI controls convergence in the entropic and Fisher information central limit theorems; and (iii) the LSI is stable with respect to HWI jumps (i.e., a jump in any of the convolution inequalities associated to the HWI functionals).
Another consequence is that the convolution inequalities for Fisher information and entropy powers are reversible in general, up to a factor depending on the Stam defect. An improved form of Nelson's hypercontractivity estimate also follows. Finally, we speculate on the possibility of an analogous reverse Brunn-Minkowski inequality and a related upper bound on surface area associated to Minkowski sums.
△ Less
Submitted 18 August, 2016;
originally announced August 2016.
-
Entropy Jumps for Radially Symmetric Random Vectors
Authors:
Thomas A. Courtade
Abstract:
We establish a quantitative bound on the entropy jump associated to the sum of independent, identically distributed (IID) radially symmetric random vectors having dimension greater than one. Following the usual approach, we first consider the analogous problem of Fisher information dissipation, and then integrate along the Ornstein-Uhlenbeck semigroup to obtain an entropic inequality. In a departu…
▽ More
We establish a quantitative bound on the entropy jump associated to the sum of independent, identically distributed (IID) radially symmetric random vectors having dimension greater than one. Following the usual approach, we first consider the analogous problem of Fisher information dissipation, and then integrate along the Ornstein-Uhlenbeck semigroup to obtain an entropic inequality. In a departure from previous work, we appeal to a result by Desvillettes and Villani on entropy production associated to the Landau equation. This obviates strong regularity assumptions, such as presence of a spectral gap and log-concavity of densities, but comes at the expense of radial symmetry. As an application, we give a quantitative estimate of the deficit in the Gaussian logarithmic Sobolev inequality for radially symmetric functions.
△ Less
Submitted 3 November, 2016; v1 submitted 18 August, 2016;
originally announced August 2016.
-
Linear Regression with an Unknown Permutation: Statistical and Computational Limits
Authors:
Ashwin Pananjady,
Martin J. Wainwright,
Thomas A. Courtade
Abstract:
Consider a noisy linear observation model with an unknown permutation, based on observing $y = Π^* A x^* + w$, where $x^* \in \mathbb{R}^d$ is an unknown vector, $Π^*$ is an unknown $n \times n$ permutation matrix, and $w \in \mathbb{R}^n$ is additive Gaussian noise. We analyze the problem of permutation recovery in a random design setting in which the entries of the matrix $A$ are drawn i.i.d. fr…
▽ More
Consider a noisy linear observation model with an unknown permutation, based on observing $y = Π^* A x^* + w$, where $x^* \in \mathbb{R}^d$ is an unknown vector, $Π^*$ is an unknown $n \times n$ permutation matrix, and $w \in \mathbb{R}^n$ is additive Gaussian noise. We analyze the problem of permutation recovery in a random design setting in which the entries of the matrix $A$ are drawn i.i.d. from a standard Gaussian distribution, and establish sharp conditions on the SNR, sample size $n$, and dimension $d$ under which $Π^*$ is exactly and approximately recoverable. On the computational front, we show that the maximum likelihood estimate of $Π^*$ is NP-hard to compute, while also providing a polynomial time algorithm when $d =1$.
△ Less
Submitted 9 August, 2016;
originally announced August 2016.
-
Brascamp-Lieb Inequality and Its Reverse: An Information Theoretic View
Authors:
**gbo Liu,
Thomas A. Courtade,
Paul Cuff,
Sergio Verdu
Abstract:
We generalize a result by Carlen and Cordero-Erausquin on the equivalence between the Brascamp-Lieb inequality and the subadditivity of relative entropy by allowing for random transformations (a broadcast channel). This leads to a unified perspective on several functional inequalities that have been gaining popularity in the context of proving impossibility results. We demonstrate that the informa…
▽ More
We generalize a result by Carlen and Cordero-Erausquin on the equivalence between the Brascamp-Lieb inequality and the subadditivity of relative entropy by allowing for random transformations (a broadcast channel). This leads to a unified perspective on several functional inequalities that have been gaining popularity in the context of proving impossibility results. We demonstrate that the information theoretic dual of the Brascamp-Lieb inequality is a convenient setting for proving properties such as data processing, tensorization, convexity and Gaussian optimality. Consequences of the latter include an extension of the Brascamp-Lieb inequality allowing for Gaussian random transformations, the determination of the multivariate Wyner common information for Gaussian sources, and a multivariate version of Nelson's hypercontractivity theorem. Finally we present an information theoretic characterization of a reverse Brascamp-Lieb inequality involving a random transformation (a multiple access channel).
△ Less
Submitted 9 May, 2016;
originally announced May 2016.
-
Partial DNA Assembly: A Rate-Distortion Perspective
Authors:
Ilan Shomorony,
Govinda M. Kamath,
Fei Xia,
Thomas A. Courtade,
David N. Tse
Abstract:
Earlier formulations of the DNA assembly problem were all in the context of perfect assembly; i.e., given a set of reads from a long genome sequence, is it possible to perfectly reconstruct the original sequence? In practice, however, it is very often the case that the read data is not sufficiently rich to permit unambiguous reconstruction of the original sequence. While a natural generalization o…
▽ More
Earlier formulations of the DNA assembly problem were all in the context of perfect assembly; i.e., given a set of reads from a long genome sequence, is it possible to perfectly reconstruct the original sequence? In practice, however, it is very often the case that the read data is not sufficiently rich to permit unambiguous reconstruction of the original sequence. While a natural generalization of the perfect assembly formulation to these cases would be to consider a rate-distortion framework, partial assemblies are usually represented in terms of an assembly graph, making the definition of a distortion measure challenging. In this work, we introduce a distortion function for assembly graphs that can be understood as the logarithm of the number of Eulerian cycles in the assembly graph, each of which correspond to a candidate assembly that could have generated the observed reads. We also introduce an algorithm for the construction of an assembly graph and analyze its performance on real genomes.
△ Less
Submitted 6 May, 2016;
originally announced May 2016.
-
Strengthening the Entropy Power Inequality
Authors:
Thomas A. Courtade
Abstract:
We tighten the Entropy Power Inequality (EPI) when one of the random summands is Gaussian. Our strengthening is closely connected to the concept of strong data processing for Gaussian channels and generalizes the (vector extension of) Costa's EPI. This leads to a new reverse entropy power inequality and, as a corollary, sharpens Stam's inequality relating entropy power and Fisher information. Appl…
▽ More
We tighten the Entropy Power Inequality (EPI) when one of the random summands is Gaussian. Our strengthening is closely connected to the concept of strong data processing for Gaussian channels and generalizes the (vector extension of) Costa's EPI. This leads to a new reverse entropy power inequality and, as a corollary, sharpens Stam's inequality relating entropy power and Fisher information. Applications to network information theory are given, including a short self-contained proof of the rate region for the two-encoder quadratic Gaussian source coding problem.
Our argument is based on weak convergence and a technique employed by Geng and Nair for establishing Gaussian optimality via rotational-invariance, which traces its roots to a `doubling trick' that has been successfully used in the study of functional inequalities.
△ Less
Submitted 9 February, 2016;
originally announced February 2016.
-
Smoothing Brascamp-Lieb Inequalities and Strong Converses for Common Randomness Generation
Authors:
**gbo Liu,
Thomas A. Courtade,
Paul Cuff,
Sergio Verdu
Abstract:
We study the infimum of the best constant in a functional inequality, the Brascamp-Lieb-like inequality, over auxiliary measures within a neighborhood of a product distribution. In the finite alphabet and the Gaussian cases, such an infimum converges to the best constant in a mutual information inequality. Implications for strong converse properties of two common randomness (CR) generation problem…
▽ More
We study the infimum of the best constant in a functional inequality, the Brascamp-Lieb-like inequality, over auxiliary measures within a neighborhood of a product distribution. In the finite alphabet and the Gaussian cases, such an infimum converges to the best constant in a mutual information inequality. Implications for strong converse properties of two common randomness (CR) generation problems are discussed. In particular, we prove the strong converse property of the rate region for the omniscient helper CR generation problem in the discrete and the Gaussian cases. The latter case is perhaps the first instance of a strong converse for a continuous source when the rate region involves auxiliary random variables.
△ Less
Submitted 6 February, 2016;
originally announced February 2016.
-
Compressing Sparse Sequences under Local Decodability Constraints
Authors:
Ashwin Pananjady,
Thomas A. Courtade
Abstract:
We consider a variable-length source coding problem subject to local decodability constraints. In particular, we investigate the blocklength scaling behavior attainable by encodings of $r$-sparse binary sequences, under the constraint that any source bit can be correctly decoded upon probing at most $d$ codeword bits. We consider both adaptive and non-adaptive access models, and derive upper and l…
▽ More
We consider a variable-length source coding problem subject to local decodability constraints. In particular, we investigate the blocklength scaling behavior attainable by encodings of $r$-sparse binary sequences, under the constraint that any source bit can be correctly decoded upon probing at most $d$ codeword bits. We consider both adaptive and non-adaptive access models, and derive upper and lower bounds that often coincide up to constant factors. Notably, such a characterization for the fixed-blocklength analog of our problem remains unknown, despite considerable research over the last three decades. Connections to communication complexity are also briefly discussed.
△ Less
Submitted 8 April, 2015;
originally announced April 2015.
-
Coded Cooperative Data Exchange for a Secret Key
Authors:
Thomas A. Courtade,
Thomas R. Halford
Abstract:
We consider a coded cooperative data exchange problem with the goal of generating a secret key. Specifically, we investigate the number of public transmissions required for a set of clients to agree on a secret key with probability one, subject to the constraint that it remains private from an eavesdropper.
Although the problems are closely related, we prove that secret key generation with fewes…
▽ More
We consider a coded cooperative data exchange problem with the goal of generating a secret key. Specifically, we investigate the number of public transmissions required for a set of clients to agree on a secret key with probability one, subject to the constraint that it remains private from an eavesdropper.
Although the problems are closely related, we prove that secret key generation with fewest number of linear transmissions is NP-hard, while it is known that the analogous problem in traditional cooperative data exchange can be solved in polynomial time. In doing this, we completely characterize the best possible performance of linear coding schemes, and also prove that linear codes can be strictly suboptimal. Finally, we extend the single-key results to characterize the minimum number of public transmissions required to generate a desired integer number of statistically independent secret keys.
△ Less
Submitted 1 July, 2014;
originally announced July 2014.
-
Outer Bounds for Multiterminal Source Coding via a Strong Data Processing Inequality
Authors:
Thomas A. Courtade
Abstract:
An intuitive outer bound for the multiterminal source coding problem is given. The proposed bound explicitly couples the rate distortion functions for each source and correlation measures which derive from a "strong" data processing inequality. Unlike many standard outer bounds, the proposed bound is not parameterized by a continuous family of auxiliary random variables, but instead only requires…
▽ More
An intuitive outer bound for the multiterminal source coding problem is given. The proposed bound explicitly couples the rate distortion functions for each source and correlation measures which derive from a "strong" data processing inequality. Unlike many standard outer bounds, the proposed bound is not parameterized by a continuous family of auxiliary random variables, but instead only requires maximizing two ratios of divergences which do not depend on the distortion functions under consideration.
△ Less
Submitted 15 July, 2013; v1 submitted 14 February, 2013;
originally announced February 2013.
-
Which Boolean Functions are Most Informative?
Authors:
Gowtham R. Kumar,
Thomas A. Courtade
Abstract:
We introduce a simply stated conjecture regarding the maximum mutual information a Boolean function can reveal about noisy inputs. Specifically, let $X^n$ be i.i.d. Bernoulli(1/2), and let $Y^n$ be the result of passing $X^n$ through a memoryless binary symmetric channel with crossover probability $α$. For any Boolean function $b:\{0,1\}^n\rightarrow \{0,1\}$, we conjecture that…
▽ More
We introduce a simply stated conjecture regarding the maximum mutual information a Boolean function can reveal about noisy inputs. Specifically, let $X^n$ be i.i.d. Bernoulli(1/2), and let $Y^n$ be the result of passing $X^n$ through a memoryless binary symmetric channel with crossover probability $α$. For any Boolean function $b:\{0,1\}^n\rightarrow \{0,1\}$, we conjecture that $I(b(X^n);Y^n)\leq 1-H(α)$. While the conjecture remains open, we provide substantial evidence supporting its validity.
△ Less
Submitted 15 July, 2013; v1 submitted 11 February, 2013;
originally announced February 2013.
-
Coded Cooperative Data Exchange in Multihop Networks
Authors:
Thomas A. Courtade,
Richard D. Wesel
Abstract:
Consider a connected network of n nodes that all wish to recover k desired packets. Each node begins with a subset of the desired packets and exchanges coded packets with its neighbors. This paper provides necessary and sufficient conditions which characterize the set of all transmission schemes that permit every node to ultimately learn (recover) all k packets. When the network satisfies certain…
▽ More
Consider a connected network of n nodes that all wish to recover k desired packets. Each node begins with a subset of the desired packets and exchanges coded packets with its neighbors. This paper provides necessary and sufficient conditions which characterize the set of all transmission schemes that permit every node to ultimately learn (recover) all k packets. When the network satisfies certain regularity conditions and packets are randomly distributed, this paper provides tight concentration results on the number of transmissions required to achieve universal recovery. For the case of a fully connected network, a polynomial-time algorithm for computing an optimal transmission scheme is derived. An application to secrecy generation is discussed.
△ Less
Submitted 15 March, 2012;
originally announced March 2012.