Showing 1–39 of 39 results for author: Motahari, A

Out-Of-Domain Unlabeled Data Improves Generalization

Authors: Amir Hossein Saberi, Amir Najafi, Alireza Heidari, Mohammad Hosein Movasaghinia, Abolfazl Motahari, Babak H. Khalaj

Abstract: We propose a novel framework for incorporating unlabeled data into semi-supervised classification problems, where scenarios involving the minimization of either i) adversarially robust or ii) non-robust loss functions have been considered. Notably, we allow the unlabeled samples to deviate slightly (in total variation sense) from the in-domain distribution. The core idea behind our framework is to… ▽ More We propose a novel framework for incorporating unlabeled data into semi-supervised classification problems, where scenarios involving the minimization of either i) adversarially robust or ii) non-robust loss functions have been considered. Notably, we allow the unlabeled samples to deviate slightly (in total variation sense) from the in-domain distribution. The core idea behind our framework is to combine Distributionally Robust Optimization (DRO) with self-supervised training. As a result, we also leverage efficient polynomial-time algorithms for the training stage. From a theoretical standpoint, we apply our framework on the classification problem of a mixture of two Gaussians in $\mathbb{R}^d$, where in addition to the $m$ independent and labeled samples from the true distribution, a set of $n$ (usually with $n\gg m$) out of domain and unlabeled samples are given as well. Using only the labeled data, it is known that the generalization error can be bounded by $\propto\left(d/m\right)^{1/2}$. However, using our method on both isotropic and non-isotropic Gaussian mixture models, one can derive a new set of analytically explicit and non-asymptotic bounds which show substantial improvement on the generalization error compared to ERM. Our results underscore two significant insights: 1) out-of-domain samples, even when unlabeled, can be harnessed to narrow the generalization gap, provided that the true data distribution adheres to a form of the ``cluster assumption", and 2) the semi-supervised learning paradigm can be regarded as a special case of our framework when there are no distributional shifts. We validate our claims through experiments conducted on a variety of synthetic and real-world datasets. △ Less

Submitted 15 February, 2024; v1 submitted 28 September, 2023; originally announced October 2023.

Comments: Published at ICLR 2024 (Spotlight), 29 pages, no figures

Sample Complexity Bounds for Learning High-dimensional Simplices in Noisy Regimes

Authors: Amir Hossein Saberi, Amir Najafi, Seyed Abolfazl Motahari, Babak H. Khalaj

Abstract: In this paper, we find a sample complexity bound for learning a simplex from noisy samples. Assume a dataset of size $n$ is given which includes i.i.d. samples drawn from a uniform distribution over an unknown simplex in $\mathbb{R}^K$, where samples are assumed to be corrupted by a multi-variate additive Gaussian noise of an arbitrary magnitude. We prove the existence of an algorithm that with hi… ▽ More In this paper, we find a sample complexity bound for learning a simplex from noisy samples. Assume a dataset of size $n$ is given which includes i.i.d. samples drawn from a uniform distribution over an unknown simplex in $\mathbb{R}^K$, where samples are assumed to be corrupted by a multi-variate additive Gaussian noise of an arbitrary magnitude. We prove the existence of an algorithm that with high probability outputs a simplex having a $\ell_2$ distance of at most $\varepsilon$ from the true simplex (for any $\varepsilon>0$). Also, we theoretically show that in order to achieve this bound, it is sufficient to have $n\ge\left(K^2/\varepsilon^2\right)e^{Ω\left(K/\mathrm{SNR}^2\right)}$ samples, where $\mathrm{SNR}$ stands for the signal-to-noise ratio. This result solves an important open problem and shows as long as $\mathrm{SNR}\geΩ\left(K^{1/2}\right)$, the sample complexity of the noisy regime has the same order to that of the noiseless case. Our proofs are a combination of the so-called sample compression technique in \citep{ashtiani2018nearly}, mathematical tools from high-dimensional geometry, and Fourier analysis. In particular, we have proposed a general Fourier-based technique for recovery of a more general class of distribution families from additive Gaussian noise, which can be further used in a variety of other related problems. △ Less

Submitted 28 April, 2023; v1 submitted 9 September, 2022; originally announced September 2022.

Comments: Accepted for ICML 2023; 27 pages

Isoform Function Prediction Using a Deep Neural Network

Authors: Sara Ghazanfari, Ali Rasteh, Seyed Abolfazl Motahari, Mahdieh Soleymani Baghshah

Abstract: Isoforms are mRNAs produced from the same gene site in the phenomenon called Alternative Splicing. Studies have shown that more than 95% of human multi-exon genes have undergone alternative splicing. Although there are few changes in mRNA sequence, They may have a systematic effect on cell function and regulation. It is widely reported that isoforms of a gene have distinct or even contrasting func… ▽ More Isoforms are mRNAs produced from the same gene site in the phenomenon called Alternative Splicing. Studies have shown that more than 95% of human multi-exon genes have undergone alternative splicing. Although there are few changes in mRNA sequence, They may have a systematic effect on cell function and regulation. It is widely reported that isoforms of a gene have distinct or even contrasting functions. Most studies have shown that alternative splicing plays a significant role in human health and disease. Despite the wide range of gene function studies, there is little information about isoforms' functionalities. Recently, some computational methods based on Multiple Instance Learning have been proposed to predict isoform function using gene function and gene expression profile. However, their performance is not desirable due to the lack of labeled training data. In addition, probabilistic models such as Conditional Random Field (CRF) have been used to model the relation between isoforms. This project uses all the data and valuable information such as isoform sequences, expression profiles, and gene ontology graphs and proposes a comprehensive model based on Deep Neural Networks. The UniProt Gene Ontology (GO) database is used as a standard reference for gene functions. The NCBI RefSeq database is used for extracting gene and isoform sequences, and the NCBI SRA database is used for expression profile data. Metrics such as Receiver Operating Characteristic Area Under the Curve (ROC AUC) and Precision-Recall Under the Curve (PR AUC) are used to measure the prediction accuracy. △ Less

Submitted 25 April, 2023; v1 submitted 5 August, 2022; originally announced August 2022.

Comments: It needs a final review from co-authors

Distributed Sparse Feature Selection in Communication-Restricted Networks

Authors: Hanie Barghi, Amir Najafi, Seyed Abolfazl Motahari

Abstract: This paper aims to propose and theoretically analyze a new distributed scheme for sparse linear regression and feature selection. The primary goal is to learn the few causal features of a high-dimensional dataset based on noisy observations from an unknown sparse linear model. However, the presumed training set which includes $n$ data samples in $\mathbb{R}^p$ is already distributed over a large n… ▽ More This paper aims to propose and theoretically analyze a new distributed scheme for sparse linear regression and feature selection. The primary goal is to learn the few causal features of a high-dimensional dataset based on noisy observations from an unknown sparse linear model. However, the presumed training set which includes $n$ data samples in $\mathbb{R}^p$ is already distributed over a large network with $N$ clients connected through extremely low-bandwidth links. Also, we consider the asymptotic configuration of $1\ll N\ll n\ll p$. In order to infer the causal dimensions from the whole dataset, we propose a simple, yet effective method for information sharing in the network. In this regard, we theoretically show that the true causal features can be reliably recovered with negligible bandwidth usage of $O\left(N\log p\right)$ across the network. This yields a significantly lower communication cost in comparison with the trivial case of transmitting all the samples to a single node (centralized scenario), which requires $O\left(np\right)$ transmissions. Even more sophisticated schemes such as ADMM still have a communication complexity of $O\left(Np\right)$. Surprisingly, our sample complexity bound is proved to be the same (up to a constant factor) as the optimal centralized approach for a fixed performance measure in each node, while that of a naïve decentralized technique grows linearly with $N$. Theoretical guarantees in this paper are based on the recent analytic framework of debiased LASSO in Javanmard et al. (2019), and are supported by several computer experiments performed on both synthetic and real-world datasets. △ Less

Submitted 2 November, 2021; originally announced November 2021.

Comments: Submitted to IEEE Transactions on Signal Processing, 14 pages

Recursive Refinement Network for Deformable Lung Registration between Exhale and Inhale CT Scans

Authors: Xinzi He, Jia Guo, Xuzhe Zhang, Hanwen Bi, Sarah Gerard, David Kaczka, Amin Motahari, Eric Hoffman, Joseph Reinhardt, R. Graham Barr, Elsa Angelini, Andrew Laine

Abstract: Unsupervised learning-based medical image registration approaches have witnessed rapid development in recent years. We propose to revisit a commonly ignored while simple and well-established principle: recursive refinement of deformation vector fields across scales. We introduce a recursive refinement network (RRN) for unsupervised medical image registration, to extract multi-scale features, const… ▽ More Unsupervised learning-based medical image registration approaches have witnessed rapid development in recent years. We propose to revisit a commonly ignored while simple and well-established principle: recursive refinement of deformation vector fields across scales. We introduce a recursive refinement network (RRN) for unsupervised medical image registration, to extract multi-scale features, construct normalized local cost correlation volume and recursively refine volumetric deformation vector fields. RRN achieves state of the art performance for 3D registration of expiratory-inspiratory pairs of CT lung scans. On DirLab COPDGene dataset, RRN returns an average Target Registration Error (TRE) of 0.83 mm, which corresponds to a 13% error reduction from the best result presented in the leaderboard. In addition to comparison with conventional methods, RRN leads to 89% error reduction compared to deep-learning-based peer approaches. △ Less

Submitted 14 June, 2021; originally announced June 2021.

Regularizing Recurrent Neural Networks via Sequence Mixup

Authors: Armin Karamzade, Amir Najafi, Seyed Abolfazl Motahari

Abstract: In this paper, we extend a class of celebrated regularization techniques originally proposed for feed-forward neural networks, namely Input Mixup (Zhang et al., 2017) and Manifold Mixup (Verma et al., 2018), to the realm of Recurrent Neural Networks (RNN). Our proposed methods are easy to implement and have a low computational complexity, while leverage the performance of simple neural architectur… ▽ More In this paper, we extend a class of celebrated regularization techniques originally proposed for feed-forward neural networks, namely Input Mixup (Zhang et al., 2017) and Manifold Mixup (Verma et al., 2018), to the realm of Recurrent Neural Networks (RNN). Our proposed methods are easy to implement and have a low computational complexity, while leverage the performance of simple neural architectures in a variety of tasks. We have validated our claims through several experiments on real-world datasets, and also provide an asymptotic theoretical analysis to further investigate the properties and potential impacts of our proposed techniques. Applying sequence mixup to BiLSTM-CRF model (Huang et al., 2015) to Named Entity Recognition task on CoNLL-2003 data (Sang and De Meulder, 2003) has improved the F-1 score on the test stage and reduced the loss, considerably. △ Less

Submitted 27 November, 2020; originally announced December 2020.

Comments: 17 pages

Private Shotgun DNA Sequencing: A Structured Approach

Authors: Ali Gholami, Mohammad Ali Maddah-Ali, Seyed Abolfazl Motahari

Abstract: DNA sequencing has faced a huge demand since it was first introduced as a service to the public. This service is often offloaded to the sequencing companies who will have access to full knowledge of individuals' sequences, a major violation of privacy. To address this challenge, we propose a solution, which is based on separating the process of reading the fragments of sequences, which is done at… ▽ More DNA sequencing has faced a huge demand since it was first introduced as a service to the public. This service is often offloaded to the sequencing companies who will have access to full knowledge of individuals' sequences, a major violation of privacy. To address this challenge, we propose a solution, which is based on separating the process of reading the fragments of sequences, which is done at a sequencing machine, and assembling the reads, which is done at a trusted local data collector. To confuse the sequencer, in a pooled sequencing scenario, in which multiple sequences are going to be sequenced simultaneously, for each target individual, we add fragments of one non-target individual, with a known DNA sequence at the data collector. Then coverage depth of the individuals, defined as the number of DNA fragments per DNA site, are selected proportional to the powers of two. This layered structured solution allows us to ensure privacy, using only one sequencing machine, in contrast to our previous solution, where we relied on the existence of multiple non-colluding sequencing machines. △ Less

Submitted 2 April, 2019; v1 submitted 28 March, 2019; originally announced April 2019.

Comments: 10 pages, 3 figures. arXiv admin note: text overlap with arXiv:1811.10693

ACM Class: E.4; H.1.1

Structure Learning of Sparse GGMs over Multiple Access Networks

Authors: Mostafa Tavassolipour, Armin Karamzade, Reza Mirzaeifard, Seyed Abolfazl Motahari, Mohammad-Taghi Manzuri Shalmani

Abstract: A central machine is interested in estimating the underlying structure of a sparse Gaussian Graphical Model (GGM) from datasets distributed across multiple local machines. The local machines can communicate with the central machine through a wireless multiple access channel. In this paper, we are interested in designing effective strategies where reliable learning is feasible under power and bandw… ▽ More A central machine is interested in estimating the underlying structure of a sparse Gaussian Graphical Model (GGM) from datasets distributed across multiple local machines. The local machines can communicate with the central machine through a wireless multiple access channel. In this paper, we are interested in designing effective strategies where reliable learning is feasible under power and bandwidth limitations. Two approaches are proposed: Signs and Uncoded methods. In Signs method, the local machines quantize their data into binary vectors and an optimal channel coding scheme is used to reliably send the vectors to the central machine where the structure is learned from the received data. In Uncoded method, data symbols are scaled and transmitted through the channel. The central machine uses the received noisy symbols to recover the structure. Theoretical results show that both methods can recover the structure with high probability for large enough sample size. Experimental results indicate the superiority of Signs method over Uncoded method under several circumstances. △ Less

Submitted 26 December, 2018; originally announced December 2018.

Cache-Aided Combination Networks with Interference

Authors: Ahmed Roushdy, Abolfazl Seyed Motahari, Mohammed Nafie, Deniz Gunduz

Abstract: Centralized coded caching and delivery is studied for a radio access combination network (RACN), whereby a set of $H$ edge nodes (ENs), connected to a cloud server via orthogonal fronthaul links with limited capacity, serve a total of $K$ user equipments (UEs) over wireless links. Each user, equipped with a cache of size $μ_R N F$ bits, is connected to a distinct set of $r$ ENs each of which equip… ▽ More Centralized coded caching and delivery is studied for a radio access combination network (RACN), whereby a set of $H$ edge nodes (ENs), connected to a cloud server via orthogonal fronthaul links with limited capacity, serve a total of $K$ user equipments (UEs) over wireless links. Each user, equipped with a cache of size $μ_R N F$ bits, is connected to a distinct set of $r$ ENs each of which equipped with a cache of size $μ_T N F$ bits, where $μ_T$, $μ_R \in [0,1]$ are the fractional cache capacities of the UEs and the ENs, respectively. The objective is to minimize the normalized delivery time (NDT. Three coded caching and transmission schemes are considered, namely the\textit{ MDS-IA}, \textit{soft-transfer} and \textit{ zero-forcing (ZF)} schemes. The achievable NDT for MDS-IA scheme is presented for $r=2$ and arbitrary fractional cache sizes $μ_T$ and $μ_R$, and also for arbitrary value of $r$ and fractional cache size $μ_T$ when the cache capacity of the UE is above a certain threshold. The achievable NDT for the soft-transfer scheme is presented for arbitrary $r$ and arbitrary fractional cache sizes $μ_T$ and $μ_R$. The last scheme utilizes ZF between the ENs and the UEs without the participation of the cloud server in the delivery phase. The achievable NDT for this scheme is presented for an arbitrary value of $r$ when the total cache size at a pair of UE and EN is sufficient to store the whole library, i.e., $μ_T+μ_R \geq 1$. The results indicate that the fronthaul capacity determines which scheme achieves a better performance in terms of the NDT, and the soft-transfer scheme becomes favorable as the fronthaul capacity increases. △ Less

Submitted 28 September, 2019; v1 submitted 15 December, 2018; originally announced December 2018.

Comments: Submitted for Publication. arXiv admin note: substantial text overlap with arXiv:1802.09087

Information Theoretic Bounds on Optimal Worst-case Error in Binary Mixture Identification

Authors: Khashayar Gatmiry, Seyed Abolfazl Motahari

Abstract: Identification of latent binary sequences from a pool of noisy observations has a wide range of applications in both statistical learning and population genetics. Each observed sequence is the result of passing one of the latent mother-sequences through a binary symmetric channel, which makes this configuration analogous to a special case of Bernoulli Mixture Models. This paper aims to attain an a… ▽ More Identification of latent binary sequences from a pool of noisy observations has a wide range of applications in both statistical learning and population genetics. Each observed sequence is the result of passing one of the latent mother-sequences through a binary symmetric channel, which makes this configuration analogous to a special case of Bernoulli Mixture Models. This paper aims to attain an asymptotically tight upper-bound on the error of Maximum Likelihood mixture identification in such problems. The obtained results demonstrate fundamental guarantees on the inference accuracy of the optimal estimator. To this end, we set out to find the closest pair of discrete distributions with respect to the Chernoff Information measure. We provide a novel technique to lower bound the Chernoff Information in an efficient way. We also show that a drastic phase transition occurs at noise level 0.25. Our findings reveal that the identification problem becomes much harder as the noise probability exceeds this threshold. △ Less

Submitted 27 November, 2018; v1 submitted 18 November, 2018; originally announced November 2018.

On Statistical Learning of Simplices: Unmixing Problem Revisited

Authors: Amir Najafi, Saeed Ilchi, Amir H. Saberi, Seyed Abolfazl Motahari, Babak H. Khalaj, Hamid R. Rabiee

Abstract: We study the sample complexity of learning a high-dimensional simplex from a set of points uniformly sampled from its interior. Learning of simplices is a long studied problem in computer science and has applications in computational biology and remote sensing, mostly under the name of `spectral unmixing'. We theoretically show that a sufficient sample complexity for reliable learning of a $K$-dim… ▽ More We study the sample complexity of learning a high-dimensional simplex from a set of points uniformly sampled from its interior. Learning of simplices is a long studied problem in computer science and has applications in computational biology and remote sensing, mostly under the name of `spectral unmixing'. We theoretically show that a sufficient sample complexity for reliable learning of a $K$-dimensional simplex up to a total-variation error of $ε$ is $O\left(\frac{K^2}ε\log\frac{K}ε\right)$, which yields a substantial improvement over existing bounds. Based on our new theoretical framework, we also propose a heuristic approach for the inference of simplices. Experimental results on synthetic and real-world datasets demonstrate a comparable performance for our method on noiseless samples, while we outperform the state-of-the-art in noisy cases. △ Less

Submitted 12 August, 2020; v1 submitted 17 October, 2018; originally announced October 2018.

Comments: 32 pages

Learning of Tree-Structured Gaussian Graphical Models on Distributed Data under Communication Constraints

Authors: Mostafa Tavassolipour, Seyed Abolfazl Motahari, Mohammad-Taghi Manzuri Shalmani

Abstract: In this paper, learning of tree-structured Gaussian graphical models from distributed data is addressed. In our model, samples are stored in a set of distributed machines where each machine has access to only a subset of features. A central machine is then responsible for learning the structure based on received messages from the other nodes. We present a set of communication efficient strategies,… ▽ More In this paper, learning of tree-structured Gaussian graphical models from distributed data is addressed. In our model, samples are stored in a set of distributed machines where each machine has access to only a subset of features. A central machine is then responsible for learning the structure based on received messages from the other nodes. We present a set of communication efficient strategies, which are theoretically proved to convey sufficient information for reliable learning of the structure. In particular, our analyses show that even if each machine sends only the signs of its local data samples to the central node, the tree structure can still be recovered with high accuracy. Our simulation results on both synthetic and real-world datasets show that our strategies achieve a desired accuracy in inferring the underlying structure, while spending a small budget on communication. △ Less

Submitted 21 September, 2018; originally announced September 2018.

The Capacity of Associated Subsequence Retrieval

Authors: Behrooz Tahmasebi, Mohammad Ali Maddah-Ali, Seyed Abolfazl Motahari

Abstract: The objective of a genome-wide association study (GWAS) is to associate subsequences of individuals' genomes to the observable characteristics called phenotypes (e.g., high blood pressure). Motivated by the GWAS problem, in this paper we introduce the information-theoretic problem of \emph{associated subsequence retrieval}, where a dataset of $N$ (possibly high-dimensional) sequences of length… ▽ More The objective of a genome-wide association study (GWAS) is to associate subsequences of individuals' genomes to the observable characteristics called phenotypes (e.g., high blood pressure). Motivated by the GWAS problem, in this paper we introduce the information-theoretic problem of \emph{associated subsequence retrieval}, where a dataset of $N$ (possibly high-dimensional) sequences of length $G$, and their corresponding observable (binary) characteristics is given. The sequences are chosen independently and uniformly at random from $\mathcal{X}^G$, where $\mathcal{X}$ is a finite alphabet. The observable (binary) characteristic is only related to a specific unknown subsequence of length $L$ of the sequences, called \textit{associated subsequence}. For each sequence, if the associated subsequence of it belongs to a universal finite set, then it is more likely to display the observable characteristic (i.e., it is more likely that the observable characteristic is one). The goal is to retrieve the associated subsequence using a dataset of $N$ sequences and their observable characteristics. We demonstrate that as the parameters $N$, $G$, and $L$ grow, a threshold effect appears in the curve of probability of error versus the rate which is defined as ${Gh(L/G)}/{N}$, where $h(\cdot)$ is the binary entropy function. This effect allows us to define the capacity of associated subsequence retrieval. We develop an achievable scheme and a matching converse for this problem, and thus characterize its capacity in two scenarios: the zero-error-rate and the $ε$-error-rate. △ Less

Submitted 14 October, 2020; v1 submitted 10 August, 2018; originally announced August 2018.

Cell Identity Codes: Understanding Cell Identity from Gene Expression Profiles using Deep Neural Networks

Authors: Farzad Abdolhosseini, Behrooz Azarkhalili, Abbas Maazallahi, Aryan Kamal, Seyed Abolfazl Motahari, Ali Sharifi-Zarchi, Hamidreza Chitsaz

Abstract: Understanding cell identity is an important task in many biomedical areas. Expression patterns of specific marker genes have been used to characterize some limited cell types, but exclusive markers are not available for many cell types. A second approach is to use machine learning to discriminate cell types based on the whole gene expression profiles (GEPs). The accuracies of simple classification… ▽ More Understanding cell identity is an important task in many biomedical areas. Expression patterns of specific marker genes have been used to characterize some limited cell types, but exclusive markers are not available for many cell types. A second approach is to use machine learning to discriminate cell types based on the whole gene expression profiles (GEPs). The accuracies of simple classification algorithms such as linear discriminators or support vector machines are limited due to the complexity of biological systems. We used deep neural networks to analyze 1040 GEPs from 16 different human tissues and cell types. After comparing different architectures, we identified a specific structure of deep autoencoders that can encode a GEP into a vector of 30 numeric values, which we call the cell identity code (CIC). The original GEP can be reproduced from the CIC with an accuracy comparable to technical replicates of the same experiment. Although we use an unsupervised approach to train the autoencoder, we show different values of the CIC are connected to different biological aspects of the cell, such as different pathways or biological processes. This network can use CIC to reproduce the GEP of the cell types it has never seen during the training. It also can resist some noise in the measurement of the GEP. Furthermore, we introduce classifier autoencoder, an architecture that can accurately identify cell type based on the GEP or the CIC. △ Less

Submitted 13 June, 2018; originally announced June 2018.

Cache-Aided Fog Radio Access Networks with Partial Connectivity

Authors: Ahmed Roushdy, Abolfazl Seyed Motahari, Mohammed Nafie, Deniz Gunduz

Abstract: Centralized coded caching and delivery is studied for a partially-connected fog radio access network (F-RAN), whereby a set of H edge nodes (ENs) (without caches), connected to a cloud server via orthogonal fronthaul links, serve K users over the wireless edge. The cloud server is assumed to hold a library of N files, each of size F bits; and each user, equipped with a cache of size MF bits, is co… ▽ More Centralized coded caching and delivery is studied for a partially-connected fog radio access network (F-RAN), whereby a set of H edge nodes (ENs) (without caches), connected to a cloud server via orthogonal fronthaul links, serve K users over the wireless edge. The cloud server is assumed to hold a library of N files, each of size F bits; and each user, equipped with a cache of size MF bits, is connected to a distinct set of r ENs; or equivalently, the wireless edge from the ENs to the users is modeled as a partial interference channel. The objective is to minimize the normalized delivery time (NDT), which refers to the worst case delivery latency, when each user requests a single file from the library. An achievable coded caching and transmission scheme is proposed, which utilizes maximum distance separable (MDS) codes in the placement phase, and real interference alignment (IA) in the delivery phase, and its achievable NDT is presented for r = 2 and arbitrary cache size M, and also for arbitrary values of r when the cache capacity is sufficiently large. △ Less

Submitted 25 February, 2018; originally announced February 2018.

Comments: To appear at the 2018 IEEE Wireless Communications and Networking Conference (WCNC)

Reliable Clustering of Bernoulli Mixture Models

Authors: Amir Najafi, Abolfazl Motahari, Hamid R. Rabiee

Abstract: A Bernoulli Mixture Model (BMM) is a finite mixture of random binary vectors with independent dimensions. The problem of clustering BMM data arises in a variety of real-world applications, ranging from population genetics to activity analysis in social networks. In this paper, we analyze the clusterability of BMMs from a theoretical perspective, when the number of clusters is unknown. In particula… ▽ More A Bernoulli Mixture Model (BMM) is a finite mixture of random binary vectors with independent dimensions. The problem of clustering BMM data arises in a variety of real-world applications, ranging from population genetics to activity analysis in social networks. In this paper, we analyze the clusterability of BMMs from a theoretical perspective, when the number of clusters is unknown. In particular, we stipulate a set of conditions on the sample complexity and dimension of the model in order to guarantee the Probably Approximately Correct (PAC)-clusterability of a dataset. To the best of our knowledge, these findings are the first non-asymptotic bounds on the sample complexity of learning or clustering BMMs. △ Less

Submitted 16 June, 2019; v1 submitted 5 October, 2017; originally announced October 2017.

Comments: 22 pages

Learning of Gaussian Processes in Distributed and Communication Limited Systems

Authors: Mostafa Tavassolipour, Seyed Abolfazl Motahari, Mohammad-Taghi Manzuri Shalmani

Abstract: It is of fundamental importance to find algorithms obtaining optimal performance for learning of statistical models in distributed and communication limited systems. Aiming at characterizing the optimal strategies, we consider learning of Gaussian Processes (GPs) in distributed systems as a pivotal example. We first address a very basic problem: how many bits are required to estimate the inner-pro… ▽ More It is of fundamental importance to find algorithms obtaining optimal performance for learning of statistical models in distributed and communication limited systems. Aiming at characterizing the optimal strategies, we consider learning of Gaussian Processes (GPs) in distributed systems as a pivotal example. We first address a very basic problem: how many bits are required to estimate the inner-products of Gaussian vectors across distributed machines? Using information theoretic bounds, we obtain an optimal solution for the problem which is based on vector quantization. Two suboptimal and more practical schemes are also presented as substitute for the vector quantization scheme. In particular, it is shown that the performance of one of the practical schemes which is called per-symbol quantization is very close to the optimal one. Schemes provided for the inner-product calculations are incorporated into our proposed distributed learning methods for GPs. Experimental results show that with spending few bits per symbol in our communication scheme, our proposed methods outperform previous zero rate distributed GP learning schemes such as Bayesian Committee Model (BCM) and Product of experts (PoE). △ Less

Submitted 7 May, 2017; originally announced May 2017.

Fundamental Limits of Pooled-DNA Sequencing

Authors: Amir Najafi, Damoun Nashta-ali, Seyed Abolfazl Motahari, Mehrdad Khani, Babak H. Khalaj, Hamid R. Rabiee

Abstract: In this paper, fundamental limits in sequencing of a set of closely related DNA molecules are addressed. This problem is called pooled-DNA sequencing which encompasses many interesting problems such as haplotype phasing, metageomics, and conventional pooled-DNA sequencing in the absence of tagging. From an information theoretic point of view, we have proposed fundamental limits on the number and l… ▽ More In this paper, fundamental limits in sequencing of a set of closely related DNA molecules are addressed. This problem is called pooled-DNA sequencing which encompasses many interesting problems such as haplotype phasing, metageomics, and conventional pooled-DNA sequencing in the absence of tagging. From an information theoretic point of view, we have proposed fundamental limits on the number and length of DNA reads in order to achieve a reliable assembly of all the pooled DNA sequences. In particular, pooled-DNA sequencing from both noiseless and noisy reads are investigated in this paper. In the noiseless case, necessary and sufficient conditions on perfect assembly are derived. Moreover, asymptotically tight lower and upper bounds on the error probability of correct assembly are obtained under a biologically plausible probabilistic model. For the noisy case, we have proposed two novel DNA read denoising methods, as well as corresponding upper bounds on assembly error probabilities. It has been shown that, under mild circumstances, the performance of the reliable assembly converges to that of the noiseless regime when, for a given read length, the number of DNA reads is sufficiently large. Interestingly, the emergence of long DNA read technologies in recent years envisions the applicability of our results in real-world applications. △ Less

Submitted 19 April, 2016; v1 submitted 16 April, 2016; originally announced April 2016.

Comments: 39 pages, Submitted to IEEE Transactions on Information Theory

Multi-Server Coded Caching

Authors: Seyed Pooya Shariatpanahi, Seyed Abolfazl Motahari, Babak Hossein Khalaj

Abstract: In this paper, we consider multiple cache-enabled clients connected to multiple servers through an intermediate network. We design several topology-aware coding strategies for such networks. Based on topology richness of the intermediate network, and types of coding operations at internal nodes, we define three classes of networks, namely, dedicated, flexible, and linear networks. For each class,… ▽ More In this paper, we consider multiple cache-enabled clients connected to multiple servers through an intermediate network. We design several topology-aware coding strategies for such networks. Based on topology richness of the intermediate network, and types of coding operations at internal nodes, we define three classes of networks, namely, dedicated, flexible, and linear networks. For each class, we propose an achievable coding scheme, analyze its coding delay, and also, compare it with an information theoretic lower bound. For flexible networks, we show that our scheme is order-optimal in terms of coding delay and, interestingly, the optimal memory-delay curve is achieved in certain regimes. In general, our results suggest that, in case of networks with multiple servers, type of network topology can be exploited to reduce service delay. △ Less

Submitted 1 March, 2015; originally announced March 2015.

Layered Interference Alignment: Achieving the Total DoF of MIMO X Channels

Authors: Seyyed Hassan Mahboubi, Mumtaz Hussain, Abolfazl Seyed Motahari, Amir Keyvan Khandani

Abstract: The $K\times 2$ and $2\times K$, Multiple-Input Multiple-Output (MIMO) X channel with constant channel coefficients available at all transmitters and receivers is considered. A new alignment scheme, named \emph{layered interference alignment}, is proposed in which both vector and real interference alignment are exploited, in conjunction with joint processing at receiver sides. Data streams with fr… ▽ More The $K\times 2$ and $2\times K$, Multiple-Input Multiple-Output (MIMO) X channel with constant channel coefficients available at all transmitters and receivers is considered. A new alignment scheme, named \emph{layered interference alignment}, is proposed in which both vector and real interference alignment are exploited, in conjunction with joint processing at receiver sides. Data streams with fractional multiplexing gains are sent in the desired directions to align the interfering signals at receivers. To decode the intended messages at receivers, a joint processing/simultaneous decoding technique, which exploits the availability of several receive antennas, is proposed. This analysis is subsequently backed up by metrical results for systems of linear forms. In particular, for such linear forms, Khintchine--Groshev type theorems are proved over real and complex numbers. It is observed that $K\times 2$ and $2\times K$, X channels with $M$ antennas at all transmitters/receivers enjoy duality in Degrees of Freedom (DoF). It is shown that incorporating the layered interference alignment is essential to characterize the total DoF of $\frac{2KM}{K+1}$ in the $K\times 2$ and $2\times K$, $M$ antenna X channels. △ Less

Submitted 22 December, 2014; originally announced December 2014.

Comments: 26 pages, 4 figures. Preliminary version. Any comments and suggestions for improvement are most welcome

Optimal DNA shotgun sequencing: Noisy reads are as good as noiseless reads

Authors: Abolfazl Motahari, Kannan Ramchandran, David Tse, Nan Ma

Abstract: We establish the fundamental limits of DNA shotgun sequencing under noisy reads. We show a surprising result: for the i.i.d. DNA model, noisy reads are as good as noiseless reads, provided that the noise level is below a certain threshold which can be surprisingly high. As an example, for a uniformly distributed DNA sequence and a symmetric substitution noisy read channel, the threshold is as high… ▽ More We establish the fundamental limits of DNA shotgun sequencing under noisy reads. We show a surprising result: for the i.i.d. DNA model, noisy reads are as good as noiseless reads, provided that the noise level is below a certain threshold which can be surprisingly high. As an example, for a uniformly distributed DNA sequence and a symmetric substitution noisy read channel, the threshold is as high as 19%. △ Less

Submitted 9 April, 2013; originally announced April 2013.

Comments: Submitted to ISIT 2013

Information Theory of DNA Shotgun Sequencing

Authors: Abolfazl Motahari, Guy Bresler, David Tse

Abstract: DNA sequencing is the basic workhorse of modern day biology and medicine. Shotgun sequencing is the dominant technique used: many randomly located short fragments called reads are extracted from the DNA sequence, and these reads are assembled to reconstruct the original sequence. A basic question is: given a sequencing technology and the statistics of the DNA sequence, what is the minimum number o… ▽ More DNA sequencing is the basic workhorse of modern day biology and medicine. Shotgun sequencing is the dominant technique used: many randomly located short fragments called reads are extracted from the DNA sequence, and these reads are assembled to reconstruct the original sequence. A basic question is: given a sequencing technology and the statistics of the DNA sequence, what is the minimum number of reads required for reliable reconstruction? This number provides a fundamental limit to the performance of {\em any} assembly algorithm. For a simple statistical model of the DNA sequence and the read process, we show that the answer admits a critical phenomena in the asymptotic limit of long DNA sequences: if the read length is below a threshold, reconstruction is impossible no matter how many reads are observed, and if the read length is above the threshold, having enough reads to cover the DNA sequence is sufficient to reconstruct. The threshold is computed in terms of the Renyi entropy rate of the DNA sequence. We also study the impact of noise in the read process on the performance. △ Less

Submitted 14 February, 2013; v1 submitted 28 March, 2012; originally announced March 2012.

Comments: Revised Version

Interference Alignment for the MIMO Interference Channel with Delayed Local CSIT

Authors: Akbar Ghasemi, Abolfazl Seyed Motahari, Amir Keyvan Khandani

Abstract: We consider the MIMO (multiple-input multiple-output) Gaussian interference channel with i.i.d. fading across antennas and channel uses and with the delayed local channel state information at the transmitters (CSIT). For the two-user case, achievability results for the degrees of freedom (DoF) region of this channel are provided. We also prove the tightness of our achievable DoF region for some an… ▽ More We consider the MIMO (multiple-input multiple-output) Gaussian interference channel with i.i.d. fading across antennas and channel uses and with the delayed local channel state information at the transmitters (CSIT). For the two-user case, achievability results for the degrees of freedom (DoF) region of this channel are provided. We also prove the tightness of our achievable DoF region for some antenna configurations. Interestingly, there are some cases in which the DoF region with delayed local CSIT is identical to the DoF region with perfect CSIT and that is strictly larger than the DoF region with no CSIT. We then consider the $K$-user MISO (multiple-input single-output) IC and show that the degrees of freedom of this channel could be greater than one with delayed local CSIT. △ Less

Submitted 28 February, 2011; originally announced February 2011.

Comments: 36 pages, 5 figures

The Approximate Capacity Region of the Gaussian Z-Interference Channel with Conferencing Encoders

Authors: Hossein Bagheri, Abolfazl S. Motahari, Amir K. Khandani

Abstract: A two-user Gaussian Z-Interference Channel (GZIC) is considered, in which encoders are connected through noiseless links with finite capacities. In this setting, prior to each transmission block the encoders communicate with each other over the cooperative links. The capacity region and the sum-capacity of the channel are characterized within 1.71 bits per user and 2 bits in total, respectively. I… ▽ More A two-user Gaussian Z-Interference Channel (GZIC) is considered, in which encoders are connected through noiseless links with finite capacities. In this setting, prior to each transmission block the encoders communicate with each other over the cooperative links. The capacity region and the sum-capacity of the channel are characterized within 1.71 bits per user and 2 bits in total, respectively. It is also established that properly sharing the total limited cooperation capacity between the cooperative links may enhance the achievable region, even when compared to the case of unidirectional transmitter cooperation with infinite cooperation capacity. To obtain the results, genie-aided upper bounds on the sum-capacity and cut-set bounds on the individual rates are compared with the achievable rate region. In the interference-limited regime, the achievable scheme enjoys a simple type of Han-Kobayashi signaling, together with the zero-forcing, and basic relaying techniques. In the noise-limited regime, it is shown that treating interference as noise achieves the capacity region up to a single bit per user. △ Less

Submitted 26 May, 2010; v1 submitted 10 May, 2010; originally announced May 2010.

Comments: 25 pages, 6 figures, submitted to IEEE Transactions on Information Theory

On the Secure Degrees-of-Freedom of the Multiple-Access-Channel

Authors: Ghadamali Bagherikaram, Abolfazl S. Motahari, Amir K. Khandani

Abstract: A $K$-user secure Gaussian Multiple-Access-Channel (MAC) with an external eavesdropper is considered in this paper. An achievable rate region is established for the secure discrete memoryless MAC. The secrecy sum capacity of the degraded Gaussian MIMO MAC is proven using Gaussian codebooks. For the non-degraded Gaussian MIMO MAC, an algorithm inspired by interference alignment technique is propo… ▽ More A $K$-user secure Gaussian Multiple-Access-Channel (MAC) with an external eavesdropper is considered in this paper. An achievable rate region is established for the secure discrete memoryless MAC. The secrecy sum capacity of the degraded Gaussian MIMO MAC is proven using Gaussian codebooks. For the non-degraded Gaussian MIMO MAC, an algorithm inspired by interference alignment technique is proposed to achieve the largest possible total Secure-Degrees-of-Freedom (S-DoF). When all the terminals are equipped with a single antenna, Gaussian codebooks have shown to be inefficient in providing a positive S-DoF. Instead, a novel secure coding scheme is proposed to achieve a positive S-DoF in the single antenna MAC. This scheme converts the single-antenna system into a multiple-dimension system with fractional dimensions. The achievability scheme is based on the alignment of signals into a small sub-space at the eavesdropper, and the simultaneous separation of the signals at the intended receiver. Tools from the field of Diophantine Approximation in number theory are used to analyze the probability of error in the coding scheme. It is proven that the total S-DoF of $\frac{K-1}{K}$ can be achieved for almost all channel gains. For the other channel gains, a multi-layer coding scheme is proposed to achieve a positive S-DoF. As a function of channel gains, therefore, the achievable S-DoF is discontinued. △ Less

Submitted 3 March, 2010; originally announced March 2010.

Comments: The conference version of this work has been submitted to ISIT 2010

Real Interference Alignment

Authors: Abolfazl Seyed Motahari, Shahab Oveis-Gharan, Mohammad-Ali Maddah-Ali, Amir Keyvan Khandani

Abstract: In this paper, we show that the total Degrees-Of-Freedoms (DOF) of the $K$-user Gaussian Interference Channel (GIC) can be achieved by incorporating a new alignment technique known as \emph{real interference alignment}. This technique compared to its ancestor \emph{vector interference alignment} performs on a single real line and exploits the properties of real numbers to provide optimal signali… ▽ More In this paper, we show that the total Degrees-Of-Freedoms (DOF) of the $K$-user Gaussian Interference Channel (GIC) can be achieved by incorporating a new alignment technique known as \emph{real interference alignment}. This technique compared to its ancestor \emph{vector interference alignment} performs on a single real line and exploits the properties of real numbers to provide optimal signaling. The real interference alignment relies on a new coding scheme in which several data streams having fractional multiplexing gains are sent by transmitters and interfering streams are aligned at receivers. The coding scheme is backed up by a recent result in the field of Diophantine approximation, which states that the convergence part of the Khintchine-Groshev theorem holds for points on non-degenerate manifolds. △ Less

Submitted 19 January, 2010; originally announced January 2010.

Comments: Submitted to ISIT 2010. For the full version, please visit arXiv:0908.2282

On the Secure DoF of the Single-Antenna MAC

Authors: Ghadamali Bagherikaram, Abolfazl S. Motahari, Amir K. Khandani

Abstract: A new achievability rate region for the secure discrete memoryless Multiple-Access-Channel (MAC) is presented. Thereafter, a novel secure coding scheme is proposed to achieve a positive Secure Degrees-of-Freedom (S-DoF) in the single-antenna MAC. This scheme converts the single-antenna system into a multiple-dimension system with fractional dimensions. The achievability scheme is based on the al… ▽ More A new achievability rate region for the secure discrete memoryless Multiple-Access-Channel (MAC) is presented. Thereafter, a novel secure coding scheme is proposed to achieve a positive Secure Degrees-of-Freedom (S-DoF) in the single-antenna MAC. This scheme converts the single-antenna system into a multiple-dimension system with fractional dimensions. The achievability scheme is based on the alignment of signals into a small sub-space at the eavesdropper, and the simultaneous separation of the signals at the intended receiver. Tools from the field of Diophantine Approximation in number theory are used to analyze the probability of error in the coding scheme. △ Less

Submitted 13 January, 2010; v1 submitted 11 January, 2010; originally announced January 2010.

Comments: 5 Pages, Submitted to ISIT 2010

On the Capacity of the Half-Duplex Diamond Channel

Authors: Hossein Bagheri, Abolfazl S. Motahari, Amir K. Khandani

Abstract: In this paper, a dual-hop communication system composed of a source S and a destination D connected through two non-interfering half-duplex relays, R1 and R2, is considered. In the literature of Information Theory, this configuration is known as the diamond channel. In this setup, four transmission modes are present, namely: 1) S transmits, and R1 and R2 listen (broadcast mode), 2) S transmits,… ▽ More In this paper, a dual-hop communication system composed of a source S and a destination D connected through two non-interfering half-duplex relays, R1 and R2, is considered. In the literature of Information Theory, this configuration is known as the diamond channel. In this setup, four transmission modes are present, namely: 1) S transmits, and R1 and R2 listen (broadcast mode), 2) S transmits, R1 listens, and simultaneously, R2 transmits and D listens. 3) S transmits, R2 listens, and simultaneously, R1 transmits and D listens. 4) R1, R2 transmit, and D listens (multiple-access mode). Assuming a constant power constraint for all transmitters, a parameter $Δ$ is defined, which captures some important features of the channel. It is proven that for $Δ$=0 the capacity of the channel can be attained by successive relaying, i.e, using modes 2 and 3 defined above in a successive manner. This strategy may have an infinite gap from the capacity of the channel when $Δ\neq$0. To achieve rates as close as 0.71 bits to the capacity, it is shown that the cases of $Δ$>0 and $Δ$<0 should be treated differently. Using new upper bounds based on the dual problem of the linear program associated with the cut-set bounds, it is proven that the successive relaying strategy needs to be enhanced by an additional broadcast mode (mode 1), or multiple access mode (mode 4), for the cases of $Δ$<0 and $Δ$>0, respectively. Furthermore, it is established that under average power constraints the aforementioned strategies achieve rates as close as 3.6 bits to the capacity of the channel. △ Less

Submitted 9 November, 2009; originally announced November 2009.

Comments: 25 pages, 2 figures, submitted to IEEE Transactions on Information Theory

Secrecy Rate Region of the Broadcast Channel with an Eavesdropper

Authors: Ghadamali Bagherikaram, Abolfazl S. Motahari, Amir K. Khandani

Abstract: In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages to two receivers, while a wire-tapper also receives the transmitted signal. This model is motivated by wireless communications, where individual secure messages are broadcast over open media and can be received by any illegitimate receiver. The secrecy level is measured by the equivocation rat… ▽ More In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages to two receivers, while a wire-tapper also receives the transmitted signal. This model is motivated by wireless communications, where individual secure messages are broadcast over open media and can be received by any illegitimate receiver. The secrecy level is measured by the equivocation rate at the eavesdropper. We first study the general (non-degraded) broadcast channel with an eavesdropper. We present an inner bound on the secrecy capacity region for this model. This inner bound is based on a combination of random binning, and the Gelfand-Pinsker binning. We further study the situation in which the channels are degraded. For the degraded broadcast channel with an eavesdropper, we present the secrecy capacity region. Our achievable coding scheme is based on Covers superposition scheme and random binning. We refer to this scheme as the Secret Superposition Scheme. Our converse proof is based on a combination of the converse proof of the conventional degraded broadcast channel and Csiszar Lemma. We then assume that the channels are Additive White Gaussian Noise (AWGN) and show that the Secret Superposition Scheme with Gaussian codebook is optimal. The converse proof is based on Costas entropy power inequality. Finally, we use a broadcast strategy for the slowly fading wire-tap channel when only the eavesdroppers channel is fixed and known at the transmitter. We derive the optimum power allocation for the coding layers, which maximizes the total average rate. △ Less

Submitted 19 October, 2009; originally announced October 2009.

Comments: 18 Pages, Submitted to IEEE Transaction on Information Theory (Revised Version)

Interference Alignment for the $K$ User MIMO Interference Channel

Authors: Akbar Ghasemi, Abolfazl Seyed Motahari, Amir Keyvan Khandani

Abstract: We consider the $K$-user Multiple Input Multiple Output (MIMO) Gaussian interference channel with $M$ antennas at each transmitter and $N$ antennas at each receiver. It is assumed that channel coefficients are constant and are available at all transmitters and at all receivers. The main objective of this paper is to characterize the Degrees of Freedom (DoF) for this channel. Using a new interferen… ▽ More We consider the $K$-user Multiple Input Multiple Output (MIMO) Gaussian interference channel with $M$ antennas at each transmitter and $N$ antennas at each receiver. It is assumed that channel coefficients are constant and are available at all transmitters and at all receivers. The main objective of this paper is to characterize the Degrees of Freedom (DoF) for this channel. Using a new interference alignment technique which has been recently introduced in \cite{abolfazl-final}, we show that $\frac{MN}{M+N} K$ degrees of freedom can be achieved for almost all channel realizations. Also, a new upper-bound on the DoF of this channel is provided. This upper-bound coincides with our achievable DoF for $K\geq K_u\define\frac{M+N}{\gcd(M,N)}$, where $\gcd(M,N)$ denotes the greatest common divisor of $M$ and $N$. This gives an exact characterization of DoF for $M\times N$ MIMO Gaussian interference channel in the case of $K\geq K_u$. △ Less

Submitted 21 September, 2011; v1 submitted 25 September, 2009; originally announced September 2009.

Comments: 25 pages, 4 figures, Presented in part at ISIT 2010, submitted to IEEE Transactions on Information Theory, June 2011

On the Symmetric Gaussian Interference Channel with Partial Unidirectional Cooperation

Authors: Hossein Bagheri, Abolfazl S. Motahari, Amir K. Khandani

Abstract: A two-user symmetric Gaussian Interference Channel (IC) is considered in which a noiseless unidirectional link connects one encoder to the other. Having a constant capacity, the additional link provides partial cooperation between the encoders. It is shown that the available cooperation can dramatically increase the sum-capacity of the channel. This fact is proved based on comparison of proposed… ▽ More A two-user symmetric Gaussian Interference Channel (IC) is considered in which a noiseless unidirectional link connects one encoder to the other. Having a constant capacity, the additional link provides partial cooperation between the encoders. It is shown that the available cooperation can dramatically increase the sum-capacity of the channel. This fact is proved based on comparison of proposed lower and upper bounds on the sum-capacity. Partitioning the data into three independent messages, namely private, common, and cooperative ones, the transmission strategy used to obtain the lower bound enjoys a simple type of Han-Kobayashi scheme together with a cooperative communication scheme. A Genie-aided upper bound is developed which incorporates the capacity of the cooperative link. Other upper bounds are based on the sum-capacity of the Cognitive Radio Channel and cut-set bounds. For the strong interference regime, the achievablity scheme is simplified to employ common and/or cooperative messages but not the private one. Through a careful analysis it is shown that the gap between these bounds is at most one and two bits per real dimension for strong and weak interference regimes, respectively. Moreover, the Generalized Degrees-of-Freedom of the channel is characterized. △ Less

Submitted 15 September, 2009; originally announced September 2009.

Real Interference Alignment: Exploiting the Potential of Single Antenna Systems

Authors: Abolfazl Seyed Motahari, Shahab Oveis Gharan, Mohammad-Ali Maddah-Ali, Amir Keyvan Khandani

Abstract: In this paper, the available spatial Degrees-Of-Freedoms (DOF) in single antenna systems is exploited. A new coding scheme is proposed in which several data streams having fractional multiplexing gains are sent by transmitters and interfering streams are aligned at receivers. Viewed as a field over rational numbers, a received signal has infinite fractional DOFs, allowing simultaneous interferen… ▽ More In this paper, the available spatial Degrees-Of-Freedoms (DOF) in single antenna systems is exploited. A new coding scheme is proposed in which several data streams having fractional multiplexing gains are sent by transmitters and interfering streams are aligned at receivers. Viewed as a field over rational numbers, a received signal has infinite fractional DOFs, allowing simultaneous interference alignment of any finite number of signals at any finite number of receivers. The coding scheme is backed up by a recent result in the field of Diophantine approximation, which states that the convergence part of the Khintchine-Groshev theorem holds for points on non-degenerate manifolds. The proposed coding scheme is proved to be optimal for three communication channels, namely the Gaussian Interference Channel (GIC), the uplink channel in cellular systems, and the $X$ channel. It is proved that the total DOF of the $K$-user GIC is $\frac{K}{2}$ almost surely, i.e. each user enjoys half of its maximum DOF. Having $K$ cells and $M$ users within each cell in a cellular system, the total DOF of the uplink channel is proved to be $\frac{KM}{M+1}$. Finally, the total DOF of the $X$ channel with $K$ transmitters and $M$ receivers is shown to be $\frac{KM}{K+M-1}$. △ Less

Submitted 23 November, 2009; v1 submitted 17 August, 2009; originally announced August 2009.

Comments: Submitted to IEEE Transaction on Information Theory. The first version was uploaded on arxiv on 17 Aug 2009 with the following title: Forming Pseudo-MIMO by Embedding Infinite Rational Dimensions Along a Single Real Line: Removing Barriers in Achieving the DOFs of Single Antenna Systems

Real Interference Alignment with Real Numbers

Authors: Abolfazl S. Motahari, Shahab Oveis Gharan, Amir K. Khandani

Abstract: A novel coding scheme applicable in networks with single antenna nodes is proposed. This scheme converts a single antenna system to an equivalent Multiple Input Multiple Output (MIMO) system with fractional dimensions. Interference can be aligned along these dimensions and higher Multiplexing gains can be achieved. Tools from the field of Diophantine approximation in number theory are used to sh… ▽ More A novel coding scheme applicable in networks with single antenna nodes is proposed. This scheme converts a single antenna system to an equivalent Multiple Input Multiple Output (MIMO) system with fractional dimensions. Interference can be aligned along these dimensions and higher Multiplexing gains can be achieved. Tools from the field of Diophantine approximation in number theory are used to show that the proposed coding scheme in fact mimics the traditional schemes used in MIMO systems where each data stream is sent along a direction and alignment happens when several streams arrive at the same direction. Two types of constellation are proposed for the encoding part, namely the single layer constellation and the multi-layer constellation. Using the single layer constellation, the coding scheme is applied to the two-user $X$ channel and the three-user Gaussian Interference Channel (GIC). In case of the two-user $X$ channel, it is proved that the total Degrees-of-Freedom (DOF), i.e. 4/3, of the channel is achievable almost surely. This is the first example in which it is shown that a time invariant single antenna system does not fall short of achieving its total DOF. Using the multi-layer constellation, the coding scheme is applied to the symmetric three-user GIC. Achievable DOFs are derived for all channel gains. As a function of the channel gain, it is observed that the DOF is everywhere discontinuous. △ Less

Submitted 12 August, 2009; v1 submitted 9 August, 2009; originally announced August 2009.

Comments: Submitted to IEEE Trans. on Information Theory

The Secrecy Capacity Region of the Degraded Vector Gaussian Broadcast Channel

Authors: Ghadamali Bagherikaram, Abolfazl S. Motahari, Amir K. Khandani

Abstract: In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages for two respective receivers via a Gaussian MIMO broadcast channel. A wire-tapper also receives the transmitted signal via another MIMO channel. It is assumed that the channels are degraded and the wire-tapper has the worst channel. We establish the capacity region of this scenario. Our achiev… ▽ More In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages for two respective receivers via a Gaussian MIMO broadcast channel. A wire-tapper also receives the transmitted signal via another MIMO channel. It is assumed that the channels are degraded and the wire-tapper has the worst channel. We establish the capacity region of this scenario. Our achievability scheme is a combination of the superposition of Gaussian codes and randomization within the layers which we will refer to as Secret Superposition Coding. For the outerbound, we use the notion of enhanced channel to show that the secret superposition of Gaussian codes is optimal. It is shown that we only need to enhance the channels of the legitimate receivers, and the channel of the eavesdropper remains unchanged. △ Less

Submitted 29 April, 2009; originally announced April 2009.

Comments: 5 Pages, 1 Figure, To be Presented at the ISIT 2009

The Secrecy Capacity Region of the Gaussian MIMO Broadcast Channel

Authors: Ghadamali Bagherikaram, Abolfazl S. Motahari, Amir K. Khandani

Abstract: In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages for two respective receivers via a Gaussian MIMO broadcast channel. A wire-tapper also receives the transmitted signal via another MIMO channel. First we assumed that the channels are degraded and the wire-tapper has the worst channel. We establish the capacity region of this scenario. Our ach… ▽ More In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages for two respective receivers via a Gaussian MIMO broadcast channel. A wire-tapper also receives the transmitted signal via another MIMO channel. First we assumed that the channels are degraded and the wire-tapper has the worst channel. We establish the capacity region of this scenario. Our achievability scheme is a combination of the superposition of Gaussian codes and randomization within the layers which we will refer to as Secret Superposition Coding. For the outerbound, we use the notion of enhanced channel to show that the secret superposition of Gaussian codes is optimal. We show that we only need to enhance the channels of the legitimate receivers, and the channel of the eavesdropper remains unchanged. Then we extend the result of the degraded case to non-degraded case. We show that the secret superposition of Gaussian codes along with successive decoding cannot work when the channels are not degraded. we develop a Secret Dirty Paper Coding (SDPC) scheme and show that SDPC is optimal for this channel. Finally, we investigate practical characterizations for the specific scenario in which the transmitter and the eavesdropper have multiple antennas, while both intended receivers have a single antenna. We characterize the secrecy capacity region in terms of generalized eigenvalues of the receivers channel and the eavesdropper channel. We refer to this configuration as the MISOME case. In high SNR we show that the capacity region is a convex closure of two rectangular regions. △ Less

Submitted 19 October, 2009; v1 submitted 18 March, 2009; originally announced March 2009.

Comments: 23 pages, 2 figures

The Secrecy Rate Region of the Broadcast Channel

Authors: Ghadamali Bagherikaram, Abolfazl S. Motahari, Amir K. Khandani

Abstract: In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages for two respective receivers, while a wire-tapper also receives the transmitted signal. This model is motivated by wireless communications, where individual secure messages are broadcast over open media and can be received by any illegitimate receiver. The secrecy level is measured by equivoca… ▽ More In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages for two respective receivers, while a wire-tapper also receives the transmitted signal. This model is motivated by wireless communications, where individual secure messages are broadcast over open media and can be received by any illegitimate receiver. The secrecy level is measured by equivocation rate at the eavesdropper. We first study the general (non-degraded) broadcast channel with confidential messages. We present an inner bound on the secrecy capacity region for this model. The inner bound coding scheme is based on a combination of random binning and the Gelfand-Pinsker bining. This scheme matches the Marton's inner bound on the broadcast channel without confidentiality constraint. We further study the situation where the channels are degraded. For the degraded broadcast channel with confidential messages, we present the secrecy capacity region. Our achievable coding scheme is based on Cover's superposition scheme and random binning. We refer to this scheme as Secret Superposition Scheme. In this scheme, we show that randomization in the first layer increases the secrecy rate of the second layer. This capacity region matches the capacity region of the degraded broadcast channel without security constraint. It also matches the secrecy capacity for the conventional wire-tap channel. Our converse proof is based on a combination of the converse proof of the conventional degraded broadcast channel and Csiszar lemma. Finally, we assume that the channels are Additive White Gaussian Noise (AWGN) and show that secret superposition scheme with Gaussian codebook is optimal. The converse proof is based on the generalized entropy power inequality. △ Less

Submitted 30 July, 2008; v1 submitted 25 June, 2008; originally announced June 2008.

Comments: 12 pages, 4 figures

On the Capacity of the Diamond Half-Duplex Relay Channel

Authors: Hossein Bagheri, Abolfazl S. Motahari, Amir K. Khandani

Abstract: We consider a diamond-shaped dual-hop communication system consisting a source, two parallel half-duplex relays and a destination. In a single antenna configuration, it has been previously shown that a two-phase node-scheduling algorithm, along with the decode and forward strategy can achieve the capacity of the diamond channel for a certain symmetric channel gains [1]. In this paper, we obtain… ▽ More We consider a diamond-shaped dual-hop communication system consisting a source, two parallel half-duplex relays and a destination. In a single antenna configuration, it has been previously shown that a two-phase node-scheduling algorithm, along with the decode and forward strategy can achieve the capacity of the diamond channel for a certain symmetric channel gains [1]. In this paper, we obtain a more general condition for the optimality of the scheme in terms of power resources and channel gains. In particular, it is proved that if the product of the capacity of the simultaneously active links are equal in both transmission phases, the scheme achieves the capacity of the channel. △ Less

Submitted 16 May, 2008; originally announced May 2008.

Capacity Bounds for the Gaussian Interference Channel

Authors: Abolfazl S. Motahari, Amir K. Khandani

Abstract: The capacity region of the two-user Gaussian Interference Channel (IC) is studied. Three classes of channels are considered: weak, one-sided, and mixed Gaussian IC. For the weak Gaussian IC, a new outer bound on the capacity region is obtained that outperforms previously known outer bounds. The sum capacity for a certain range of channel parameters is derived. For this range, it is proved that u… ▽ More The capacity region of the two-user Gaussian Interference Channel (IC) is studied. Three classes of channels are considered: weak, one-sided, and mixed Gaussian IC. For the weak Gaussian IC, a new outer bound on the capacity region is obtained that outperforms previously known outer bounds. The sum capacity for a certain range of channel parameters is derived. For this range, it is proved that using Gaussian codebooks and treating interference as noise is optimal. It is shown that when Gaussian codebooks are used, the full Han-Kobayashi achievable rate region can be obtained by using the naive Han-Kobayashi achievable scheme over three frequency bands (equivalently, three subspaces). For the one-sided Gaussian IC, an alternative proof for the Sato's outer bound is presented. We derive the full Han-Kobayashi achievable rate region when Gaussian codebooks are utilized. For the mixed Gaussian IC, a new outer bound is obtained that outperforms previously known outer bounds. For this case, the sum capacity for the entire range of channel parameters is derived. It is proved that the full Han-Kobayashi achievable rate region using Gaussian codebooks is equivalent to that of the one-sided Gaussian IC for a particular range of channel parameters. △ Less

Submitted 8 January, 2008; originally announced January 2008.

Comments: 35 pages, 14 figures, submitted to IEEE Trans. on Inf. Theory

To Decode the Interference or To Consider it as Noise

Authors: Abolfazl S. Motahari, Amir K. Khandani

Abstract: We address single-user data transmission over a channel where the received signal incurs interference from a finite number of users (interfering users) that use single codebooks for transmitting their own messages. The receiver, however, is allowed to decode interfering users' messages. This means the signal transmitted from any interfering user is either decoded or considered as noise at the re… ▽ More We address single-user data transmission over a channel where the received signal incurs interference from a finite number of users (interfering users) that use single codebooks for transmitting their own messages. The receiver, however, is allowed to decode interfering users' messages. This means the signal transmitted from any interfering user is either decoded or considered as noise at the receiver side. We propose the following method to obtain an achievable rate for this channel. Assuming its own data is decoded successfully, the receiver partitions the set of interfering users into two disjoint subsets, namely the set of decodable users and the set of non-decodable users. Then the transmitter's rate is chosen such that the intended signal can be jointly decoded with the set of decodable users. To show the strength of this method, we prove that for the additive Gaussian channel with Gaussian interfering users, the Gaussian distribution is optimal and the achievable rate is the capacity of this channel. To obtain the maximum achievable rate, one needs to find the maximum decodable subset of interfering users. Due to the large number of possible choices, having efficient algorithms that find the set of decodable users with maximum cardinality is desired. To this end, we propose an algorithm that enables the receiver to accomplish this task in polynomial time. △ Less

Submitted 20 November, 2007; originally announced November 2007.

Comments: submitted to IEEE Transactions on Information Theory