-
Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding
Authors:
Yigit Ugur,
George Arvanitakis,
Abdellatif Zaidi
Abstract:
In this paper, we develop an unsupervised generative clustering framework that combines the Variational Information Bottleneck and the Gaussian Mixture Model. Specifically, in our approach, we use the Variational Information Bottleneck method and model the latent space as a mixture of Gaussians. We derive a bound on the cost function of our model that generalizes the Evidence Lower Bound (ELBO) an…
▽ More
In this paper, we develop an unsupervised generative clustering framework that combines the Variational Information Bottleneck and the Gaussian Mixture Model. Specifically, in our approach, we use the Variational Information Bottleneck method and model the latent space as a mixture of Gaussians. We derive a bound on the cost function of our model that generalizes the Evidence Lower Bound (ELBO) and provide a variational inference type algorithm that allows computing it. In the algorithm, the coders' map**s are parametrized using neural networks, and the bound is approximated by Monte Carlo sampling and optimized with stochastic gradient descent. Numerical results on real datasets are provided to support the efficiency of our method.
△ Less
Submitted 27 February, 2020; v1 submitted 28 May, 2019;
originally announced May 2019.
-
Vector Gaussian CEO Problem Under Logarithmic Loss
Authors:
Yigit Ugur,
Inaki Estella Aguerri,
Abdellatif Zaidi
Abstract:
In this paper, we study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO wants to reconstruct the remote source to within so…
▽ More
In this paper, we study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion. We find an explicit characterization of the rate-distortion region of this model. For the proof of this result, we obtain an outer bound on the region of the vector Gaussian CEO problem by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, we also show that the established result under logarithmic loss provides an outer bound for a quadratic vector Gaussian CEO problem with determinant constraint, for which we characterize the optimal rate-distortion region.
△ Less
Submitted 25 February, 2019;
originally announced February 2019.
-
Vector Gaussian CEO Problem Under Logarithmic Loss and Applications
Authors:
Yigit Ugur,
Inaki Estella Aguerri,
Abdellatif Zaidi
Abstract:
We study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO also has its own Gaussian noisy observation of the source and want…
▽ More
We study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO also has its own Gaussian noisy observation of the source and wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion. We find an explicit characterization of the rate-distortion region of this model. The result can be seen as the counterpart to the vector Gaussian setting of that by Courtade-Weissman which provides the rate-distortion region of the model in the discrete memoryless setting. For the proof of this result, we obtain an outer bound by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, application of our results allows us to find the complete solutions of two related problems: a quadratic vector Gaussian CEO problem with determinant constraint and the vector Gaussian distributed Information Bottleneck problem. Finally, we develop Blahut-Arimoto type algorithms that allow to compute numerically the regions provided in this paper, for both discrete and Gaussian models. We illustrate the efficiency of our algorithms through some numerical examples.
△ Less
Submitted 4 February, 2020; v1 submitted 9 November, 2018;
originally announced November 2018.
-
A Generalization of Blahut-Arimoto Algorithm to Compute Rate-Distortion Regions of Multiterminal Source Coding Under Logarithmic Loss
Authors:
Yigit Ugur,
Inaki Estella Aguerri,
Abdellatif Zaidi
Abstract:
In this paper, we present iterative algorithms that numerically compute the rate-distortion regions of two problems: the two-encoder multiterminal source coding problem and the Chief Executive Officer (CEO) problem, both under logarithmic loss distortion measure.With the clear connection of these models with the distributed information bottleneck method, the proposed algorithms may find usefulness…
▽ More
In this paper, we present iterative algorithms that numerically compute the rate-distortion regions of two problems: the two-encoder multiterminal source coding problem and the Chief Executive Officer (CEO) problem, both under logarithmic loss distortion measure.With the clear connection of these models with the distributed information bottleneck method, the proposed algorithms may find usefulness in a variety of applications, such as clustering, pattern recognition and learning. We illustrate the efficiency of our algorithms through some numerical examples.
△ Less
Submitted 24 August, 2017;
originally announced August 2017.
-
Reducing MIMO Detection Complexity via Hierarchical Modulation
Authors:
Yigit Ugur,
Ali Ozgur Yilmaz
Abstract:
This work considers multiple-input multiple-output (MIMO) communication systems using hierarchical modulation. A disadvantage of the maximum-likelihood (ML) MIMO detector is that computational complexity increases exponentially with the number of transmit antennas. To reduce complexity, we propose a hierarchical modulation scheme to be used in MIMO trans- mission where base and enhancement layers…
▽ More
This work considers multiple-input multiple-output (MIMO) communication systems using hierarchical modulation. A disadvantage of the maximum-likelihood (ML) MIMO detector is that computational complexity increases exponentially with the number of transmit antennas. To reduce complexity, we propose a hierarchical modulation scheme to be used in MIMO trans- mission where base and enhancement layers are incorporated. In the proposed receiver, the base layer is detected first with a minimum mean square error (MMSE) detector which is followed by ML detection of the enhancement layer. Our results indicate that the proposed low complexity scheme does not compromise performance when design parameters such as code rates and constellation ratio are chosen carefully.
△ Less
Submitted 23 February, 2016;
originally announced February 2016.
-
Cloud Radio Access Networks with Coded Caching
Authors:
Yigit Ugur,
Zohaib Hassan Awan,
Aydin Sezgin
Abstract:
A cloud radio access network (C-RAN) is considered as a candidate to meet the expectations of higher data rate de- mands in wireless networks. In C-RAN, low energy base stations (BSs) are deployed over a small geography and are allowed to connect to the cloud via finite capacity backhaul links where the information is processed. A conventional C-RAN, however, requires high capacity backhaul links,…
▽ More
A cloud radio access network (C-RAN) is considered as a candidate to meet the expectations of higher data rate de- mands in wireless networks. In C-RAN, low energy base stations (BSs) are deployed over a small geography and are allowed to connect to the cloud via finite capacity backhaul links where the information is processed. A conventional C-RAN, however, requires high capacity backhaul links, since the requested files need to be transferred first from the cloud to the BS before conveying them to the users. One approach to overcome the limitations of the backhaul links is to introduce local storage caches at the BSs, in which the popular files are stored locally in order to reduce the load of the backhaul links. Furthermore, we utilize coded caching with the goal to minimize the total network cost, i.e., the transmit power and the cost associated with the backhaul links. The initial formulation of the optimization problem for this model is non-convex. We first reformulate and then convexify the problem through some relaxation techniques. In comparison to the uncoded caching at the BSs, our results highlight the benefits associated with coded caching and show that it decreases the backhaul cost.
△ Less
Submitted 10 February, 2016; v1 submitted 8 December, 2015;
originally announced December 2015.
