Showing 1–15 of 15 results for author: Aguerri, I E

Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

Authors: Deniz Gunduz, Zhi** Qin, Inaki Estella Aguerri, Harpreet S. Dhillon, Zhaohui Yang, Aylin Yener, Kai Kit Wong, Chan-Byoung Chae

Abstract: Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these… ▽ More Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications. △ Less

Submitted 3 October, 2022; v1 submitted 19 July, 2022; originally announced July 2022.

Comments: 32 pages, 14 figures

On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Views

Authors: Abdellatif Zaidi, Inaki Estella Aguerri, Shlomo Shamai

Abstract: This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote source-coding under logarithmic loss distortion measure, information combining, common reconstruction, the Wyner-Ahlswede-Korner pro… ▽ More This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote source-coding under logarithmic loss distortion measure, information combining, common reconstruction, the Wyner-Ahlswede-Korner problem, the efficiency of investment information, as well as, generalization, variational inference, representation learning, autoencoders, and others are highlighted. We discuss its extension to the distributed information bottleneck problem with emphasis on the Gaussian model and highlight the basic connections to the uplink Cloud Radio Access Networks (CRAN) with oblivious processing. For this model, the optimal trade-offs between relevance (i.e., information) and complexity (i.e., rates) in the discrete and vector Gaussian frameworks is determined. In the concluding outlook, some interesting problems are mentioned such as the characterization of the optimal inputs ("features") distributions under power limitations maximizing the "relevance" for the Gaussian information bottleneck, under "complexity" constraints. △ Less

Submitted 31 January, 2020; originally announced February 2020.

Comments: To be published in Entropy as part of the Special Issue Information Theory for Data Communications and Processing. 51 pages. arXiv admin note: text overlap with arXiv:1807.04193

Optimal Rate-Exponent Region for a Class of Hypothesis Testing Against Conditional Independence Problems

Authors: Abdellatif Zaidi, Inaki Estella Aguerri

Abstract: We study a class of distributed hypothesis testing against conditional independence problems. Under the criterion that stipulates minimization of the Type II error rate subject to a (constant) upper bound $ε$ on the Type I error rate, we characterize the set of encoding rates and exponent for both discrete memoryless and memoryless vector Gaussian settings. We study a class of distributed hypothesis testing against conditional independence problems. Under the criterion that stipulates minimization of the Type II error rate subject to a (constant) upper bound $ε$ on the Type I error rate, we characterize the set of encoding rates and exponent for both discrete memoryless and memoryless vector Gaussian settings. △ Less

Submitted 4 April, 2019; originally announced April 2019.

Comments: Submitted for publication to the IEEE Information Theory Workshop, ITW 2019. arXiv admin note: substantial text overlap with arXiv:1811.03933

Vector Gaussian CEO Problem Under Logarithmic Loss

Authors: Yigit Ugur, Inaki Estella Aguerri, Abdellatif Zaidi

Abstract: In this paper, we study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO wants to reconstruct the remote source to within so… ▽ More In this paper, we study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion. We find an explicit characterization of the rate-distortion region of this model. For the proof of this result, we obtain an outer bound on the region of the vector Gaussian CEO problem by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, we also show that the established result under logarithmic loss provides an outer bound for a quadratic vector Gaussian CEO problem with determinant constraint, for which we characterize the optimal rate-distortion region. △ Less

Submitted 25 February, 2019; originally announced February 2019.

Comments: This paper was accepted at the IEEE Information Theory Workshop (ITW), 2018. 5 pages, 1 figure. arXiv admin note: substantial text overlap with arXiv:1811.03933

Vector Gaussian CEO Problem Under Logarithmic Loss and Applications

Authors: Yigit Ugur, Inaki Estella Aguerri, Abdellatif Zaidi

Abstract: We study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO also has its own Gaussian noisy observation of the source and want… ▽ More We study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO also has its own Gaussian noisy observation of the source and wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion. We find an explicit characterization of the rate-distortion region of this model. The result can be seen as the counterpart to the vector Gaussian setting of that by Courtade-Weissman which provides the rate-distortion region of the model in the discrete memoryless setting. For the proof of this result, we obtain an outer bound by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, application of our results allows us to find the complete solutions of two related problems: a quadratic vector Gaussian CEO problem with determinant constraint and the vector Gaussian distributed Information Bottleneck problem. Finally, we develop Blahut-Arimoto type algorithms that allow to compute numerically the regions provided in this paper, for both discrete and Gaussian models. We illustrate the efficiency of our algorithms through some numerical examples. △ Less

Submitted 4 February, 2020; v1 submitted 9 November, 2018; originally announced November 2018.

Comments: accepted for publication in IEEE Transactions on Information Theory

Distributed Variational Representation Learning

Authors: Inaki Estella Aguerri, Abdellatif Zaidi

Abstract: The problem of distributed representation learning is one in which multiple sources of information $X_1,\ldots,X_K$ are processed separately so as to learn as much information as possible about some ground truth $Y$. We investigate this problem from information-theoretic grounds, through a generalization of Tishby's centralized Information Bottleneck (IB) method to the distributed setting. Specifi… ▽ More The problem of distributed representation learning is one in which multiple sources of information $X_1,\ldots,X_K$ are processed separately so as to learn as much information as possible about some ground truth $Y$. We investigate this problem from information-theoretic grounds, through a generalization of Tishby's centralized Information Bottleneck (IB) method to the distributed setting. Specifically, $K$ encoders, $K \geq 2$, compress their observations $X_1,\ldots,X_K$ separately in a manner such that, collectively, the produced representations preserve as much information as possible about $Y$. We study both discrete memoryless (DM) and memoryless vector Gaussian data models. For the discrete model, we establish a single-letter characterization of the optimal tradeoff between complexity (or rate) and relevance (or information) for a class of memoryless sources (the observations $X_1,\ldots,X_K$ being conditionally independent given $Y$). For the vector Gaussian model, we provide an explicit characterization of the optimal complexity-relevance tradeoff. Furthermore, we develop a variational bound on the complexity-relevance tradeoff which generalizes the evidence lower bound (ELBO) to the distributed setting. We also provide two algorithms that allow to compute this bound: i) a Blahut-Arimoto type iterative algorithm which enables to compute optimal complexity-relevance encoding map**s by iterating over a set of self-consistent equations, and ii) a variational inference type algorithm in which the encoding map**s are parametrized by neural networks and the bound approximated by Markov sampling and optimized with stochastic gradient descent. Numerical results on synthetic and real datasets are provided to support the efficiency of the approaches and algorithms developed in this paper. △ Less

Submitted 31 March, 2019; v1 submitted 11 July, 2018; originally announced July 2018.

Comments: 35 pages, 10 figures, submitted for possible publication

On the Capacity of Cloud Radio Access Networks with Oblivious Relaying

Authors: Inaki Estella Aguerri, Abdellatif Zaidi, Giuseppe Caire, Shlomo Shamai

Abstract: We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users' codebooks, i.e., they perform oblivious processing. The destination, or central processor, however, i… ▽ More We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users' codebooks, i.e., they perform oblivious processing. The destination, or central processor, however, is informed about the users' codebooks. We establish a single-letter characterization of the capacity region of this model for a class of discrete memoryless channels in which the outputs at the relay nodes are independent given the users' inputs. We show that both relaying à-la Cover-El Gamal, i.e., compress-and-forward with joint decompression and decoding, and "noisy network coding", are optimal. The proof of the converse part establishes, and utilizes, connections with the Chief Executive Officer (CEO) source coding problem under logarithmic loss distortion measure. Extensions to general discrete memoryless channels are also investigated. In this case, we establish inner and outer bounds on the capacity region. For memoryless Gaussian channels within the studied class of channels, we characterize the capacity region when the users are constrained to time-share among Gaussian codebooks. Furthermore, we also discuss the suboptimality of separate decompression-decoding and the role of time-sharing. △ Less

Submitted 30 January, 2019; v1 submitted 25 October, 2017; originally announced October 2017.

Comments: Accepted to IEEE Transactions on Information Theory

Distributed Information Bottleneck Method for Discrete and Gaussian Sources

Authors: Inaki Estella Aguerri, Abdellatif Zaidi

Abstract: We study the problem of distributed information bottleneck, in which multiple encoders separately compress their observations in a manner such that, collectively, the compressed signals preserve as much information as possible about another signal. The model generalizes Tishby's centralized information bottleneck method to the setting of multiple distributed encoders. We establish single-letter ch… ▽ More We study the problem of distributed information bottleneck, in which multiple encoders separately compress their observations in a manner such that, collectively, the compressed signals preserve as much information as possible about another signal. The model generalizes Tishby's centralized information bottleneck method to the setting of multiple distributed encoders. We establish single-letter characterizations of the information-rate region of this problem for both i) a class of discrete memoryless sources and ii) memoryless vector Gaussian sources. Furthermore, assuming a sum constraint on rate or complexity, for both models we develop Blahut-Arimoto type iterative algorithms that allow to compute optimal information-rate trade-offs, by iterating over a set of self-consistent equations. △ Less

Submitted 3 October, 2017; v1 submitted 26 September, 2017; originally announced September 2017.

Comments: Submitted to the 2018 International Zurich Seminar on Information and Communication (IZS)

A Generalization of Blahut-Arimoto Algorithm to Compute Rate-Distortion Regions of Multiterminal Source Coding Under Logarithmic Loss

Authors: Yigit Ugur, Inaki Estella Aguerri, Abdellatif Zaidi

Abstract: In this paper, we present iterative algorithms that numerically compute the rate-distortion regions of two problems: the two-encoder multiterminal source coding problem and the Chief Executive Officer (CEO) problem, both under logarithmic loss distortion measure.With the clear connection of these models with the distributed information bottleneck method, the proposed algorithms may find usefulness… ▽ More In this paper, we present iterative algorithms that numerically compute the rate-distortion regions of two problems: the two-encoder multiterminal source coding problem and the Chief Executive Officer (CEO) problem, both under logarithmic loss distortion measure.With the clear connection of these models with the distributed information bottleneck method, the proposed algorithms may find usefulness in a variety of applications, such as clustering, pattern recognition and learning. We illustrate the efficiency of our algorithms through some numerical examples. △ Less

Submitted 24 August, 2017; originally announced August 2017.

Comments: IEEE Information Theory Workshop (ITW) 2017. Accepted for publication

On the Capacity of Cloud Radio Access Networks with Oblivious Relaying

Authors: Inaki Estella Aguerri, Abdellatif Zaidi, Giuseppe Caire, Shlomo Shamai

Abstract: We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users' codebooks, i.e., they perform oblivious processing - The destination, or central processor, however,… ▽ More We study the transmission over a network in which users send information to a remote destination through relay nodes that are connected to the destination via finite-capacity error-free links, i.e., a cloud radio access network. The relays are constrained to operate without knowledge of the users' codebooks, i.e., they perform oblivious processing - The destination, or central processor, however, is informed about the users' codebooks. We establish a single-letter characterization of the capacity region of this model for a class of discrete memoryless channels in which the outputs at the relay nodes are independent given the users' inputs. We show that both relaying à-la Cover-El Gamal, i.e., compress-and-forward with joint decompression and decoding, and "noisy network coding", are optimal. The proof of the converse part establishes, and utilizes, connections with the Chief Executive Officer (CEO) source coding problem under logarithmic loss distortion measure. Extensions to general discrete memoryless channels are also investigated. In this case, we establish inner and outer bounds on the capacity region. For memoryless Gaussian channels within the studied class of channels, we characterize the capacity under Gaussian channel inputs. △ Less

Submitted 25 January, 2017; originally announced January 2017.

Comments: Submitted to the 2017 IEEE Int. Symposium on Information Theory (extended version, with more results, will be submitted to the IEEE Trans. on Information Theory)

In-network Compression for Multiterminal Cascade MIMO Systems

Authors: Inaki Estella Aguerri, Abdellatif Zaidi

Abstract: We study the problem of receive beamforming in uplink cascade multiple-input multiple-output (MIMO) systems as an instance of that of cascade multiterminal source coding for lossy function computation. Using this connection, we develop two coding schemes for the second and show that their application leads to beamforming schemes for the first. In the first coding scheme, each terminal in the casca… ▽ More We study the problem of receive beamforming in uplink cascade multiple-input multiple-output (MIMO) systems as an instance of that of cascade multiterminal source coding for lossy function computation. Using this connection, we develop two coding schemes for the second and show that their application leads to beamforming schemes for the first. In the first coding scheme, each terminal in the cascade sends a description of the source that it observes; the decoder reconstructs all sources, lossily, and then computes an estimate of the desired function. This scheme improves upon standard routing in that every terminal only compresses the innovation of its source w.r.t. the descriptions that are sent by the previous terminals in the cascade. In the second scheme, the desired function is computed gradually in the cascade network, and each terminal sends a finer description of it. In the context of uplink cascade MIMO systems, the application of these two schemes leads to centralized receive-beamforming and distributed receive-beamforming, respectively. Numerical results illustrate the performance of the proposed methods and show that they outperform standard routing. △ Less

Submitted 6 March, 2017; v1 submitted 5 December, 2016; originally announced December 2016.

Comments: Submitted to IEEE Transactions on Communications

Lossy Compression for Compute-and-Forward in Limited Backhaul Uplink Multicell Processing

Authors: Iñaki Estella Aguerri, Abdellatif Zaidi

Abstract: We study the transmission over a cloud radio access network in which multiple base stations (BS) are connected to a central processor (CP) via finite-capacity backhaul links. We propose two lattice-based coding schemes. In the first scheme, the base stations decode linear combinations of the transmitted messages, in the spirit of compute-and-forward (CoF), but differs from it essentially in that t… ▽ More We study the transmission over a cloud radio access network in which multiple base stations (BS) are connected to a central processor (CP) via finite-capacity backhaul links. We propose two lattice-based coding schemes. In the first scheme, the base stations decode linear combinations of the transmitted messages, in the spirit of compute-and-forward (CoF), but differs from it essentially in that the decoded equations are remapped to linear combinations of the channel input symbols, sent compressed in a lossy manner to the central processor, and are not required to be linearly independent. Also, by opposition to the standard CoF, an appropriate multi-user decoder is utilized to recover the sent messages. The second coding scheme generalizes the first one by also allowing, at each relay node, a joint compression of the decoded equation and the received signal. Both schemes apply in general, but are more suited for situations in which there are more users than base stations. We show that both schemes can outperform standard CoF and successive Wyner-Ziv schemes in certain regimes, and illustrate the gains through some numerical examples. △ Less

Submitted 28 February, 2016; originally announced February 2016.

Comments: Submitted to IEEE Transactions on Communications

Distortion Exponent in MIMO Fading Channels with Time-Varying Source Side Information

Authors: Iñaki Estella Aguerri, Deniz Gündüz

Abstract: Transmission of a Gaussian source over a time-varying multiple-input multiple-output (MIMO) channel is studied under strict delay constraints. Availability of a correlated side information at the receiver is assumed, whose quality, i.e., correlation with the source signal, also varies over time. A block-fading model is considered for the states of the time-varying channel and the time-varying side… ▽ More Transmission of a Gaussian source over a time-varying multiple-input multiple-output (MIMO) channel is studied under strict delay constraints. Availability of a correlated side information at the receiver is assumed, whose quality, i.e., correlation with the source signal, also varies over time. A block-fading model is considered for the states of the time-varying channel and the time-varying side information; and perfect state information at the receiver is assumed, while the transmitter knows only the statistics. The high SNR performance, characterized by the \textit{distortion exponent}, is studied for this joint source-channel coding problem. An upper bound is derived and compared with lowers based on list decoding, hybrid digital-analog transmission, as well as multi-layer schemes which transmit successive refinements of the source, relying on progressive and superposed transmission with list decoding. The optimal distortion exponent is characterized for the single-input multiple-output (SIMO) and multiple-input single-output (MISO) scenarios by showing that the distortion exponent achieved by multi-layer superpositon encoding with joint decoding meets the proposed upper bound. In the MIMO scenario, the optimal distortion exponent is characterized in the low bandwidth ratio regime, and it is shown that the multi-layer superposition encoding performs very close to the upper bound in the high bandwidth expansion regime. △ Less

Submitted 26 May, 2015; v1 submitted 1 September, 2014; originally announced September 2014.

Comments: Submitted to IEEE Transactions on Information Theory

Capacity of a Class of State-Dependent Orthogonal Relay Channels

Authors: Iñaki Estella Aguerri, Deniz Gündüz

Abstract: The class of orthogonal relay channels in which the orthogonal channels connecting the source terminal to the relay and the destination, and the relay to the destination, depend on a state sequence, is considered. It is assumed that the state sequence is fully known at the destination while it is not known at the source or the relay. The capacity of this class of relay channels is characterized, a… ▽ More The class of orthogonal relay channels in which the orthogonal channels connecting the source terminal to the relay and the destination, and the relay to the destination, depend on a state sequence, is considered. It is assumed that the state sequence is fully known at the destination while it is not known at the source or the relay. The capacity of this class of relay channels is characterized, and shown to be achieved by the partial decode-compress-and-forward (pDCF) scheme. Then the capacity of certain binary and Gaussian state-dependent orthogonal relay channels are studied in detail, and it is shown that the compress-and-forward (CF) and partial-decode-and-forward (pDF) schemes are suboptimal in general. To the best of our knowledge, this is the first single relay channel model for which the capacity is achieved by pDCF, while pDF and CF schemes are both suboptimal. Furthermore, it is shown that the capacity of the considered class of state-dependent orthogonal relay channels is in general below the cut-set bound. The conditions under which pDF or CF suffices to meet the cut-set bound, and hence, achieve the capacity, are also derived. △ Less

Submitted 17 December, 2015; v1 submitted 20 May, 2014; originally announced May 2014.

Comments: This paper has been accepted by IEEE Transactions on Information Theory

Joint Source-Channel Coding with Time-Varying Channel and Side-Information

Authors: Iñaki Estella Aguerri, Deniz Gündüz

Abstract: Transmission of a Gaussian source over a time-varying Gaussian channel is studied in the presence of time-varying correlated side information at the receiver. A block fading model is considered for both the channel and the side information, whose states are assumed to be known only at the receiver. The optimality of separate source and channel coding in terms of average end-to-end distortion is sh… ▽ More Transmission of a Gaussian source over a time-varying Gaussian channel is studied in the presence of time-varying correlated side information at the receiver. A block fading model is considered for both the channel and the side information, whose states are assumed to be known only at the receiver. The optimality of separate source and channel coding in terms of average end-to-end distortion is shown when the channel is static while the side information state follows a discrete or a continuous and quasiconcave distribution. When both the channel and side information states are time-varying, separate source and channel coding is suboptimal in general. A partially informed encoder lower bound is studied by providing the channel state information to the encoder. Several achievable transmission schemes are proposed based on uncoded transmission, separate source and channel coding, joint decoding as well as hybrid digital-analog transmission. Uncoded transmission is shown to be optimal for a class of continuous and quasiconcave side information state distributions, while the channel gain may have an arbitrary distribution. To the best of our knowledge, this is the first example in which the uncoded transmission achieves the optimal performance thanks to the time-varying nature of the states, while it is suboptimal in the static version of the same problem. Then, the optimal \emph{distortion exponent}, that quantifies the exponential decay rate of the expected distortion in the high SNR regime, is characterized for Nakagami distributed channel and side information states, and it is shown to be achieved by hybrid digital-analog and joint decoding schemes in certain cases, illustrating the suboptimality of pure digital or analog transmission in general. △ Less

Submitted 26 May, 2015; v1 submitted 3 December, 2013; originally announced December 2013.

Comments: Submitted to IEEE Transactions on Information Theory