Showing 1–1 of 1 results for author: de Cabrera, F
-
Regularized Estimation of Information via High Dimensional Canonical Correlation Analysis
Authors:
Jaume Riba,
Ferran de Cabrera
Abstract:
In recent years, there has been an upswing of interest in estimating information from data emerging in a lot of areas beyond communications. This paper aims at estimating the information between two random phenomena by using consolidated second-order statistics tools. The squared-loss mutual information is chosen for that purpose as a natural surrogate of Shannon mutual information. The rationale…
▽ More
In recent years, there has been an upswing of interest in estimating information from data emerging in a lot of areas beyond communications. This paper aims at estimating the information between two random phenomena by using consolidated second-order statistics tools. The squared-loss mutual information is chosen for that purpose as a natural surrogate of Shannon mutual information. The rationale for doing so is developed for i.i.d. discrete sources -map** data onto the simplex space-, and for analog sources -map** data onto the characteristic space-, highlighting the links with other well-known related concepts in the literature based on local approximations of information-theoretic measures. The proposed approach gains in interpretability and scalability for its use on large datasets, providing physical interpretation to the free regularization parameters. Moreover, the structure of the proposed map** allows resorting to Szegö's theorem to reduce the complexity for high dimensional map**s, exhibiting strong dualities with spectral analysis. The performance of the proposed estimators is analyzed using Gaussian mixtures.
△ Less
Submitted 6 May, 2020;
originally announced May 2020.
