Showing 1–4 of 4 results for author: Mirkarimi, F

A Perspective on Neural Capacity Estimation: Viability and Reliability

Authors: Farhad Mirkarimi, Stefano Rini, Nariman Farsad

Abstract: Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. These estimators ar referred to as neural mutual information estimation (NMIE)s. NMIEs differ from other approaches as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. In order to test the performance… ▽ More Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. These estimators ar referred to as neural mutual information estimation (NMIE)s. NMIEs differ from other approaches as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. In order to test the performance across various NMIEs, it is desirable to establish a benchmark encompassing the different challenges of capacity estimation. This is the objective of this paper. In particular, we consider three scenarios for benchmarking:i the classic AWGN channel, ii channels continuous inputs optical intensity and peak-power constrained AWGN channel iii channels with a discrete output, i.e., Poisson channel. We also consider the extension to the multi-terminal case with iv the AWGN and optical MAC models. We argue that benchmarking a certain NMIE across these four scenarios provides a substantive test of performance. In this paper we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE). and provide insights on the performance of other methods as well. To summarize our benchmarking results, MINE provides the most reliable performance. △ Less

Submitted 5 October, 2022; v1 submitted 22 March, 2022; originally announced March 2022.

Comments: 33 pages, 9 figures, under revison for possible journal publication

Neural Capacity Estimators: How Reliable Are They?

Authors: Farhad Mirkarimi, Stefano Rini, Nariman Farsad

Abstract: Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowing closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of impleme… ▽ More Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowing closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of implementation, and stability for capacity estimation which is joint maximization frame-work. In this paper, we compare the different techniques proposed in the literature for estimating capacity and provide a practitioner perspective on their effectiveness. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights on InfoNCE. We evaluated these algorithms in terms of their ability to learn the input distributions that are capacity approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel. For both scenarios, we provide insightful comments on various aspects of the training process, such as stability, sensitivity to initialization. △ Less

Submitted 18 March, 2022; v1 submitted 14 November, 2021; originally announced November 2021.

Comments: 7 pages, accepted for publication at the 2022 IEEE International Conference on Communications (ICC)

Neural Computation of Capacity Region of Memoryless Multiple Access Channels

Authors: Farhad Mirkarimi, Nariman Farsad

Abstract: This paper provides a numerical framework for computing the achievable rate region of memoryless multiple access channel (MAC) with a continuous alphabet from data. In particular, we use recent results on variational lower bounds on mutual information and KL-divergence to compute the boundaries of the rate region of MAC using a set of functions parameterized by neural networks. Our method relies o… ▽ More This paper provides a numerical framework for computing the achievable rate region of memoryless multiple access channel (MAC) with a continuous alphabet from data. In particular, we use recent results on variational lower bounds on mutual information and KL-divergence to compute the boundaries of the rate region of MAC using a set of functions parameterized by neural networks. Our method relies on a variational lower bound on KL-divergence and an upper bound on KL-divergence based on the f-divergence inequalities. Unlike previous work, which computes an estimate on mutual information, which is neither a lower nor an upper bound, our method estimates a lower bound on mutual information. Our numerical results show that the proposed method provides tighter estimates compared to the MINE-based estimator at large SNRs while being computationally more efficient. Finally, we apply the proposed method to the optical intensity MAC and obtain a new achievable rate boundary tighter than prior works. △ Less

Submitted 10 May, 2021; originally announced May 2021.

Comments: 6 pages, 4 figures, accepted at ISIT2021

On the Capacity of the Joint Time and Concentration Modulation for Molecular Communications

Authors: Farhad Mirkarimi, Mahtab Mirmohseni, Masoumeh Nasiri-Kenari

Abstract: Most diffusion based molecular channels suffer from low information capacity due to the structure of the diffusion environment. To address this issue, this paper studies the capacity of the diffusion based molecular communication by exploiting both time and concentration level of the released molecules for information transfer. The transmitter releases molecules in one of the sub-intervals with a… ▽ More Most diffusion based molecular channels suffer from low information capacity due to the structure of the diffusion environment. To address this issue, this paper studies the capacity of the diffusion based molecular communication by exploiting both time and concentration level of the released molecules for information transfer. The transmitter releases molecules in one of the sub-intervals with a level of concentration both determined by input data, hereby applying joint time and concentration (JTAC) modulation. The observation time, at the receiver, which is equal to symbol period, is divided to some sub-intervals, not necessarily equal to the number of sub-intervals in the transmitter, and the number of received molecules in each sub-interval is counted. We propose three practical schemes, depending on how the receiver uses the number of molecules counted in the sub-intervals and find the lower bound on capacity in each case. Moreover, the symmetric Kullback-Liebler (KL) divergence metric is used to obtain a computable upper bound on the JTAC channel capacity. Finally, the Blahut-Arimoto algorithm is used to compute the capacity numerically, and to determine how tight the derived bounds are. Our numerical results indicate that our bounds are tight especially in environments with high diffusion coefficient. The improvements compared to the conventional concentration based modulation and timing based modulation are also demonstrated. △ Less

Submitted 23 June, 2020; originally announced June 2020.