-
Adaptive Dam** and Mean Removal for the Generalized Approximate Message Passing Algorithm
Authors:
Jeremy Vila,
Philip Schniter,
Sundeep Rangan,
Florent Krzakala,
Lenka Zdeborova
Abstract:
The generalized approximate message passing (GAMP) algorithm is an efficient method of MAP or approximate-MMSE estimation of $x$ observed from a noisy version of the transform coefficients $z = Ax$. In fact, for large zero-mean i.i.d sub-Gaussian $A$, GAMP is characterized by a state evolution whose fixed points, when unique, are optimal. For generic $A$, however, GAMP may diverge. In this paper,…
▽ More
The generalized approximate message passing (GAMP) algorithm is an efficient method of MAP or approximate-MMSE estimation of $x$ observed from a noisy version of the transform coefficients $z = Ax$. In fact, for large zero-mean i.i.d sub-Gaussian $A$, GAMP is characterized by a state evolution whose fixed points, when unique, are optimal. For generic $A$, however, GAMP may diverge. In this paper, we propose adaptive dam** and mean-removal strategies that aim to prevent divergence. Numerical results demonstrate significantly enhanced robustness to non-zero-mean, rank-deficient, column-correlated, and ill-conditioned $A$.
△ Less
Submitted 5 December, 2014;
originally announced December 2014.
-
Low-Rank Spatial Channel Estimation for Millimeter Wave Cellular Systems
Authors:
Parisa A. Eliasi,
Sundeep Rangan,
Theodore S. Rappaport
Abstract:
The tremendous bandwidth available in the millimeter wave (mmW) frequencies between 30 and 300 GHz have made these bands an attractive candidate for next-generation cellular systems. However, reliable communication at these frequencies depends extensively on beamforming with very high-dimensional antenna arrays. Estimating the channel sufficiently accurately to perform beamforming can thus be chal…
▽ More
The tremendous bandwidth available in the millimeter wave (mmW) frequencies between 30 and 300 GHz have made these bands an attractive candidate for next-generation cellular systems. However, reliable communication at these frequencies depends extensively on beamforming with very high-dimensional antenna arrays. Estimating the channel sufficiently accurately to perform beamforming can thus be challenging both due to low coherence time and large number of antennas. Also, the measurements used for channel estimation may need to be made with analog beamforming where the receiver can "look" in only direction at a time. This work presents a novel method for estimation of the receive-side spatial covariance matrix of a channel from a sequence of power measurements made at different angular directions. The method reduces the spatial covariance estimation to a matrix completion optimization problem. To reduce the number of measurements, the optimization can incorporate the low-rank constraints in the channels that are typical in the mmW setting. The optimization is convex and fast, iterative methods are presented to solving the problem. Simulations are presented for both single and multi-path channels using channel models derived from real measurements in New York City at 28 GHz.
△ Less
Submitted 17 October, 2014;
originally announced October 2014.
-
Scalable Inference for Neuronal Connectivity from Calcium Imaging
Authors:
Alyson K. Fletcher,
Sundeep Rangan
Abstract:
Fluorescent calcium imaging provides a potentially powerful tool for inferring connectivity in neural circuits with up to thousands of neurons. However, a key challenge in using calcium imaging for connectivity detection is that current systems often have a temporal response and frame rate that can be orders of magnitude slower than the underlying neural spiking process. Bayesian inference methods…
▽ More
Fluorescent calcium imaging provides a potentially powerful tool for inferring connectivity in neural circuits with up to thousands of neurons. However, a key challenge in using calcium imaging for connectivity detection is that current systems often have a temporal response and frame rate that can be orders of magnitude slower than the underlying neural spiking process. Bayesian inference methods based on expectation-maximization (EM) have been proposed to overcome these limitations, but are often computationally demanding since the E-step in the EM procedure typically involves state estimation for a high-dimensional nonlinear dynamical system. In this work, we propose a computationally fast method for the state estimation based on a hybrid of loopy belief propagation and approximate message passing (AMP). The key insight is that a neural system as viewed through calcium imaging can be factorized into simple scalar dynamical systems for each neuron with linear interconnections between the neurons. Using the structure, the updates in the proposed hybrid AMP methodology can be computed by a set of one-dimensional state estimation procedures and linear transforms with the connectivity matrix. This yields a computationally scalable method for inferring connectivity of large neural circuits. Simulations of the method on realistic neural networks demonstrate good accuracy with computation times that are potentially significantly faster than current approaches based on Markov Chain Monte Carlo methods.
△ Less
Submitted 3 September, 2014; v1 submitted 1 September, 2014;
originally announced September 2014.
-
Compressive Phase Retrieval via Generalized Approximate Message Passing
Authors:
Philip Schniter,
Sundeep Rangan
Abstract:
In phase retrieval, the goal is to recover a signal $\mathbf{x}\in\mathbb{C}^N$ from the magnitudes of linear measurements $\mathbf{Ax}\in\mathbb{C}^M$. While recent theory has established that $M\approx 4N$ intensity measurements are necessary and sufficient to recover generic $\mathbf{x}$, there is great interest in reducing the number of measurements through the exploitation of sparse…
▽ More
In phase retrieval, the goal is to recover a signal $\mathbf{x}\in\mathbb{C}^N$ from the magnitudes of linear measurements $\mathbf{Ax}\in\mathbb{C}^M$. While recent theory has established that $M\approx 4N$ intensity measurements are necessary and sufficient to recover generic $\mathbf{x}$, there is great interest in reducing the number of measurements through the exploitation of sparse $\mathbf{x}$, which is known as compressive phase retrieval. In this work, we detail a novel, probabilistic approach to compressive phase retrieval based on the generalized approximate message passing (GAMP) algorithm. We then present a numerical study of the proposed PR-GAMP algorithm, demonstrating its excellent phase-transition behavior, robustness to noise, and runtime. Our experiments suggest that approximately $M\geq 2K\log_2(N/K)$ intensity measurements suffice to recover $K$-sparse Bernoulli-Gaussian signals for $\mathbf{A}$ with i.i.d Gaussian entries and $K\ll N$. Meanwhile, when recovering a 6k-sparse 65k-pixel grayscale image from 32k randomly masked and blurred Fourier intensity measurements at 30~dB measurement SNR, PR-GAMP achieved an output SNR of no less than 28~dB in all of 100 random trials, with a median runtime of only 7.3 seconds. Compared to the recently proposed CPRL, sparse-Fienup, and GESPAR algorithms, our experiments suggest that PR-GAMP has a superior phase transition and orders-of-magnitude faster runtimes as the sparsity and problem dimensions increase.
△ Less
Submitted 17 October, 2014; v1 submitted 21 May, 2014;
originally announced May 2014.
-
Scaling Laws for Infrastructure Single and Multihop Wireless Networks in Wideband Regimes
Authors:
Felipe Gomez-Cuba,
Sundeep Rangan,
Elza Erkip
Abstract:
With millimeter wave bands emerging as a strong candidate for 5G cellular networks, next-generation systems may be in a unique position where spectrum is plentiful. To assess the potential value of this spectrum, this paper derives scaling laws on the per mobile downlink feasible rate with large bandwidth and number of nodes, for both Infrastructure Single Hop (ISH) and Infrastructure Multi-Hop (I…
▽ More
With millimeter wave bands emerging as a strong candidate for 5G cellular networks, next-generation systems may be in a unique position where spectrum is plentiful. To assess the potential value of this spectrum, this paper derives scaling laws on the per mobile downlink feasible rate with large bandwidth and number of nodes, for both Infrastructure Single Hop (ISH) and Infrastructure Multi-Hop (IMH) architectures. It is shown that, for both cases, there exist \emph{critical bandwidth scalings} above which increasing the bandwidth no longer increases the feasible rate per node. These critical thresholds coincide exactly with the bandwidths where, for each architecture, the network transitions from being degrees-of-freedom-limited to power-limited. For ISH, this critical bandwidth threshold is lower than IMH when the number of users per base station grows with network size. This result suggests that multi-hop transmissions may be necessary to fully exploit large bandwidth degrees of freedom in deployments with growing number of users per cell.
△ Less
Submitted 28 April, 2014;
originally announced April 2014.
-
Directional Cell Discovery in Millimeter Wave Cellular Networks
Authors:
C. Nicolas Barati,
S. Amir Hosseini,
Sundeep Rangan,
Pei Liu,
Thanasis Korakis,
Shivendra S. Panwar,
Theodore S. Rappaport
Abstract:
The acute disparity between increasing bandwidth demand and available spectrum, has brought millimeter wave (mmW) bands to the forefront of candidate solutions for the next-generation cellular networks. Highly directional transmissions are essential for cellular communication in these frequencies to compensate for high isotropic path loss. This reliance on directional beamforming, however, complic…
▽ More
The acute disparity between increasing bandwidth demand and available spectrum, has brought millimeter wave (mmW) bands to the forefront of candidate solutions for the next-generation cellular networks. Highly directional transmissions are essential for cellular communication in these frequencies to compensate for high isotropic path loss. This reliance on directional beamforming, however, complicates initial cell search since the mobile and base station must jointly search over a potentially large angular directional space to locate a suitable path to initiate communication. To address this problem, this paper proposes a directional cell discovery procedure where base stations periodically transmit synchronization signals, potentially in time-varying random directions, to scan the angular space. Detectors for these signals are derived based on a Generalized Likelihood Ratio Test (GLRT) under various signal and receiver assumptions. The detectors are then simulated under realistic design parameters and channels based on actual experimental measurements at 28~GHz in New York City. The study reveals two key findings: (i) digital beamforming can significantly outperform analog beamforming even when the digital beamforming uses very low quantization to compensate for the additional power requirements; and (ii) omni-directional transmissions of the synchronization signals from the base station generally outperforms random directional scanning.
△ Less
Submitted 21 April, 2015; v1 submitted 20 April, 2014;
originally announced April 2014.
-
On the Convergence of Approximate Message Passing with Arbitrary Matrices
Authors:
Sundeep Rangan,
Philip Schniter,
Alyson K. Fletcher,
Subrata Sarkar
Abstract:
Approximate message passing (AMP) methods and their variants have attracted considerable recent attention for the problem of estimating a random vector $\mathbf{x}$ observed through a linear transform $\mathbf{A}$. In the case of large i.i.d. zero-mean Gaussian $\mathbf{A}$, the methods exhibit fast convergence with precise analytic characterizations on the algorithm behavior. However, the converg…
▽ More
Approximate message passing (AMP) methods and their variants have attracted considerable recent attention for the problem of estimating a random vector $\mathbf{x}$ observed through a linear transform $\mathbf{A}$. In the case of large i.i.d. zero-mean Gaussian $\mathbf{A}$, the methods exhibit fast convergence with precise analytic characterizations on the algorithm behavior. However, the convergence of AMP under general transforms $\mathbf{A}$ is not fully understood. In this paper, we provide sufficient conditions for the convergence of a damped version of the generalized AMP (GAMP) algorithm in the case of quadratic cost functions (i.e., Gaussian likelihood and prior). It is shown that, with sufficient dam**, the algorithm is guaranteed to converge, although the amount of dam** grows with peak-to-average ratio of the squared singular values of the transforms $\mathbf{A}$. This result explains the good performance of AMP on i.i.d. Gaussian transforms $\mathbf{A}$, but also their difficulties with ill-conditioned or non-zero-mean transforms $\mathbf{A}$. A related sufficient condition is then derived for the local stability of the damped GAMP method under general cost functions, assuming certain strict convexity conditions.
△ Less
Submitted 2 March, 2018; v1 submitted 13 February, 2014;
originally announced February 2014.
-
Millimeter Wave Cellular Wireless Networks: Potentials and Challenges
Authors:
Sundeep Rangan,
Theodore S. Rappaport,
Elza Erkip
Abstract:
Millimeter wave (mmW) frequencies between 30 and 300 GHz are a new frontier for cellular communication that offers the promise of orders of magnitude greater bandwidths combined with further gains via beamforming and spatial multiplexing from multi-element antenna arrays. This paper surveys measurements and capacity studies to assess this technology with a focus on small cell deployments in urban…
▽ More
Millimeter wave (mmW) frequencies between 30 and 300 GHz are a new frontier for cellular communication that offers the promise of orders of magnitude greater bandwidths combined with further gains via beamforming and spatial multiplexing from multi-element antenna arrays. This paper surveys measurements and capacity studies to assess this technology with a focus on small cell deployments in urban environments. The conclusions are extremely encouraging; measurements in New York City at 28 and 73 GHz demonstrate that, even in an urban canyon environment, significant non-line-of-sight (NLOS) outdoor, street-level coverage is possible up to approximately 200 m from a potential low power micro- or picocell base station. In addition, based on statistical channel models from these measurements, it is shown that mmW systems can offer more than an order of magnitude increase in capacity over current state-of-the-art 4G cellular networks at current cell densities. Cellular systems, however, will need to be significantly redesigned to fully achieve these gains. Specifically, the requirement of highly directional and adaptive transmissions, directional isolation between links and significant possibilities of outage have strong implications on multiple access, channel structure, synchronization and receiver design. To address these challenges, the paper discusses how various technologies including adaptive beamforming, multihop relaying, heterogeneous network architectures and carrier aggregation can be leveraged in the mmW context.
△ Less
Submitted 11 January, 2014;
originally announced January 2014.
-
Millimeter Wave Channel Modeling and Cellular Capacity Evaluation
Authors:
Mustafa Riza Akdeniz,
Yuanpeng Liu,
Mathew K. Samimi,
Shu Sun,
Sundeep Rangan,
Theodore S. Rappaport,
Elza Erkip
Abstract:
With the severe spectrum shortage in conventional cellular bands, millimeter wave (mmW) frequencies between 30 and 300 GHz have been attracting growing attention as a possible candidate for next-generation micro- and picocellular wireless networks. The mmW bands offer orders of magnitude greater spectrum than current cellular allocations and enable very high-dimensional antenna arrays for further…
▽ More
With the severe spectrum shortage in conventional cellular bands, millimeter wave (mmW) frequencies between 30 and 300 GHz have been attracting growing attention as a possible candidate for next-generation micro- and picocellular wireless networks. The mmW bands offer orders of magnitude greater spectrum than current cellular allocations and enable very high-dimensional antenna arrays for further gains via beamforming and spatial multiplexing. This paper uses recent real-world measurements at 28 and 73 GHz in New York City to derive detailed spatial statistical models of the channels and uses these models to provide a realistic assessment of mmW micro- and picocellular networks in a dense urban deployment. Statistical models are derived for key channel parameters including the path loss, number of spatial clusters, angular dispersion and outage. It is found that, even in highly non-line-of-sight environments, strong signals can be detected 100 m to 200 m from potential cell sites, potentially with multiple clusters to support spatial multiplexing. Moreover, a system simulation based on the models predicts that mmW systems can offer an order of magnitude increase in capacity over current state-of-the-art 4G cellular networks with no increase in cell density from current urban deployments.
△ Less
Submitted 25 April, 2014; v1 submitted 17 December, 2013;
originally announced December 2013.
-
Generalized Approximate Message Passing for Cosparse Analysis Compressive Sensing
Authors:
Mark Borgerding,
Philip Schniter,
Sundeep Rangan
Abstract:
In cosparse analysis compressive sensing (CS), one seeks to estimate a non-sparse signal vector from noisy sub-Nyquist linear measurements by exploiting the knowledge that a given linear transform of the signal is cosparse, i.e., has sufficiently many zeros. We propose a novel approach to cosparse analysis CS based on the generalized approximate message passing (GAMP) algorithm. Unlike other AMP-b…
▽ More
In cosparse analysis compressive sensing (CS), one seeks to estimate a non-sparse signal vector from noisy sub-Nyquist linear measurements by exploiting the knowledge that a given linear transform of the signal is cosparse, i.e., has sufficiently many zeros. We propose a novel approach to cosparse analysis CS based on the generalized approximate message passing (GAMP) algorithm. Unlike other AMP-based approaches to this problem, ours works with a wide range of analysis operators and regularizers. In addition, we propose a novel $\ell_0$-like soft-thresholder based on MMSE denoising for a spike-and-slab distribution with an infinite-variance slab. Numerical demonstrations on synthetic and practical datasets demonstrate advantages over existing AMP-based, greedy, and reweighted-$\ell_1$ approaches.
△ Less
Submitted 19 October, 2014; v1 submitted 13 December, 2013;
originally announced December 2013.
-
Opportunistic Third-Party Backhaul for Cellular Wireless Networks
Authors:
Russell Ford,
Changkyu Kim,
Sundeep Rangan
Abstract:
With high capacity air interfaces and large numbers of small cells, backhaul -- the wired connectivity to base stations -- is increasingly becoming the cost driver in cellular wireless networks. One reason for the high cost of backhaul is that capacity is often purchased on leased lines with guaranteed rates provisioned to peak loads. In this paper, we present an alternate \emph{opportunistic back…
▽ More
With high capacity air interfaces and large numbers of small cells, backhaul -- the wired connectivity to base stations -- is increasingly becoming the cost driver in cellular wireless networks. One reason for the high cost of backhaul is that capacity is often purchased on leased lines with guaranteed rates provisioned to peak loads. In this paper, we present an alternate \emph{opportunistic backhaul} model where third parties provide base stations and backhaul connections and lease out excess capacity in their networks to the cellular provider when available, presumably at significantly lower costs than guaranteed connections. We describe a scalable architecture for such deployments using open access femtocells, which are small plug-and-play base stations that operate in the carrier's spectrum but can connect directly into the third party provider's wired network. Within the proposed architecture, we present a general user association optimization algorithm that enables the cellular provider to dynamically determine which mobiles should be assigned to the third-party femtocells based on the traffic demands, interference and channel conditions and third-party access pricing. Although the optimization is non-convex, the algorithm uses a computationally efficient method for finding approximate solutions via dual decomposition. Simulations of the deployment model based on actual base station locations are presented that show that large capacity gains are achievable if adoption of third-party, open access femtocells can reach even a small fraction of the current market penetration of WiFi access points.
△ Less
Submitted 4 May, 2013;
originally announced May 2013.
-
Joint Interference and User Association Optimization in Cellular Wireless Networks
Authors:
Changkyu Kim,
Russell Ford,
Yanjia Qi,
Sundeep Rangan
Abstract:
In cellular wireless networks, user association refers to the problem of assigning mobile users to base station cells -- a critical, but challenging, problem in many emerging small cell and heterogeneous networks. This paper considers a general class of utility maximization problems for joint optimization of mobile user associations and bandwidth and power allocations. The formulation can incorpor…
▽ More
In cellular wireless networks, user association refers to the problem of assigning mobile users to base station cells -- a critical, but challenging, problem in many emerging small cell and heterogeneous networks. This paper considers a general class of utility maximization problems for joint optimization of mobile user associations and bandwidth and power allocations. The formulation can incorporate a large class of network topologies, interference models, SNR-to-rate map**s and network constraints. In addition, the model can applied in carrier aggregation scenarios where mobiles can be served by multiple cells simultaneously. While the problem is non-convex, our main contribution shows that the optimization admits a separable dual decomposition. This property enables fast computation of upper bounds on the utility as well as an efficient, distributed implementation for approximate local optimization via augmented Lagrangian techniques. Simulations are presented in heterogeneous networks with mixtures of macro and picocells. We demonstrate significant value of the proposed methods in scenarios with variable backhaul capacity in the femtocell links and in cases where the user density is sufficiently low that lightly-used cells can reduce power.
△ Less
Submitted 15 April, 2013;
originally announced April 2013.
-
Millimeter Wave Picocellular System Evaluation for Urban Deployments
Authors:
Mustafa Riza Akdeniz,
Yuanpeng Liu,
Sundeep Rangan,
Elza Erkip
Abstract:
With the severe spectrum shortage in conventional cellular bands, millimeter wave (mmW) frequencies between 30 and 300 GHz have been attracting growing attention as a possible candidate for next-generation micro- and picocellular wireless networks. The mmW bands offer orders of magnitude greater spectrum than current cellular allocations and enable very high-dimensional antenna arrays for further…
▽ More
With the severe spectrum shortage in conventional cellular bands, millimeter wave (mmW) frequencies between 30 and 300 GHz have been attracting growing attention as a possible candidate for next-generation micro- and picocellular wireless networks. The mmW bands offer orders of magnitude greater spectrum than current cellular allocations and enable very high-dimensional antenna arrays for further gains via spatial multiplexing. However, the propagation of mmW signals in outdoor non line-of-sight (NLOS) links remains challenging and the feasibility of wide-area mmW cellular networks is far from clear. This paper uses recent real-world measurements at 28 GHz in New York City to provide a realistic assessment of mmW picocellular networks in a dense urban deployment. It is found that, even under conservative propagation assumptions, mmW systems with cell radii of 100m can offer an order of magnitude increase in capacity over current state-of-the-art 4G cellular networks with similar cell density. However, it is also shown that such mmW networks may operate in a largely power-limited regime where the full spatial and bandwidth degrees of freedom are not fully utilized. This power-limited regime contrasts significantly with current bandwidth-limited cellular systems, requiring alternate technologies for mmW systems that may unlock further gains that mmW frequency bands offer.
△ Less
Submitted 25 April, 2014; v1 submitted 14 April, 2013;
originally announced April 2013.
-
Fixed Points of Generalized Approximate Message Passing with Arbitrary Matrices
Authors:
Sundeep Rangan,
Philip Schniter,
Erwin Riegler,
Alyson Fletcher,
Volkan Cevher
Abstract:
The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these met…
▽ More
The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed-points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed-points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.
△ Less
Submitted 2 May, 2016; v1 submitted 26 January, 2013;
originally announced January 2013.
-
Wireless Scheduling with Dominant Interferers and Applications to Femtocellular Interference Cancellation
Authors:
Mustafa Riza Akdeniz,
Sundeep Rangan
Abstract:
We consider a general class of wireless scheduling and resource allocation problems where the received rate in each link is determined by the actions of the transmitter in that link along with a single dominant interferer. Such scenarios arise in a range of scenarios, particularly in emerging femto- and picocellular networks with strong, localized interference. For these networks, a utility maximi…
▽ More
We consider a general class of wireless scheduling and resource allocation problems where the received rate in each link is determined by the actions of the transmitter in that link along with a single dominant interferer. Such scenarios arise in a range of scenarios, particularly in emerging femto- and picocellular networks with strong, localized interference. For these networks, a utility maximizing scheduler based on loopy belief propagation is presented that enables computationally-efficient local processing and low communication overhead. Our main theoretical result shows that the fixed points of the method are provably globally optimal for arbitrary (potentially non-convex) rate and utility functions. The methodology thus provides globally optimal solutions to a large class of inter-cellular interference coordination problems including subband scheduling, dynamic orthogonalization and beamforming whenever the dominant interferer assumption is valid. The paper focuses on applications for systems with interference cancellation (IC) and suggests a new scheme on optimal rate control, as opposed to traditional power control. Simulations are presented in industry standard femtocellular network models demonstrate significant improvements in rates over simple reuse 1 without IC, and near optimal performance of loopy belief propagation for rate selection in only one or two iterations.
△ Less
Submitted 29 July, 2012;
originally announced July 2012.
-
Approximate Message Passing with Consistent Parameter Estimation and Applications to Sparse Learning
Authors:
Ulugbek S. Kamilov,
Sundeep Rangan,
Alyson K. Fletcher,
Michael Unser
Abstract:
We consider the estimation of an i.i.d. (possibly non-Gaussian) vector $\xbf \in \R^n$ from measurements $\ybf \in \R^m$ obtained by a general cascade model consisting of a known linear transform followed by a probabilistic componentwise (possibly nonlinear) measurement channel. A novel method, called adaptive generalized approximate message passing (Adaptive GAMP), that enables joint learning of…
▽ More
We consider the estimation of an i.i.d. (possibly non-Gaussian) vector $\xbf \in \R^n$ from measurements $\ybf \in \R^m$ obtained by a general cascade model consisting of a known linear transform followed by a probabilistic componentwise (possibly nonlinear) measurement channel. A novel method, called adaptive generalized approximate message passing (Adaptive GAMP), that enables joint learning of the statistics of the prior and measurement channel along with estimation of the unknown vector $\xbf$ is presented. The proposed algorithm is a generalization of a recently-developed EM-GAMP that uses expectation-maximization (EM) iterations where the posteriors in the E-steps are computed via approximate message passing. The methodology can be applied to a large class of learning problems including the learning of sparse priors in compressed sensing or identification of linear-nonlinear cascade models in dynamical systems and neural spiking processes. We prove that for large i.i.d. Gaussian transform matrices the asymptotic componentwise behavior of the adaptive GAMP algorithm is predicted by a simple set of scalar state evolution equations. In addition, we show that when a certain maximum-likelihood estimation can be performed in each step, the adaptive GAMP method can yield asymptotically consistent parameter estimates, which implies that the algorithm achieves a reconstruction quality equivalent to the oracle algorithm that knows the correct parameter values. Remarkably, this result applies to essentially arbitrary parametrizations of the unknown distributions, including ones that are nonlinear and non-Gaussian. The adaptive GAMP methodology thus provides a systematic, general and computationally efficient method applicable to a large range of complex linear-nonlinear models with provable guarantees.
△ Less
Submitted 1 December, 2012; v1 submitted 16 July, 2012;
originally announced July 2012.
-
Iterative Reconstruction of Rank-One Matrices in Noise
Authors:
Alyson K. Fletcher,
Sundeep Rangan
Abstract:
We consider the problem of estimating a rank-one matrix in Gaussian noise under a probabilistic model for the left and right factors of the matrix. The probabilistic model can impose constraints on the factors including sparsity and positivity that arise commonly in learning problems. We propose a family of algorithms that reduce the problem to a sequence of scalar estimation computations. These a…
▽ More
We consider the problem of estimating a rank-one matrix in Gaussian noise under a probabilistic model for the left and right factors of the matrix. The probabilistic model can impose constraints on the factors including sparsity and positivity that arise commonly in learning problems. We propose a family of algorithms that reduce the problem to a sequence of scalar estimation computations. These algorithms are similar to approximate message passing techniques based on Gaussian approximations of loopy belief propagation that have been used recently in compressed sensing. Leveraging analysis methods by Bayati and Montanari, we show that the asymptotic behavior of the algorithm is described by a simple scalar equivalent model, where the distribution of the estimates at each iteration is identical to certain scalar estimates of the variables in Gaussian noise. Moreover, the effective Gaussian noise level is described by a set of state evolution equations. The proposed approach to deriving algorithms thus provides a computationally simple and general method for rank-one estimation problems with a precise analysis in certain high-dimensional settings.
△ Less
Submitted 15 September, 2015; v1 submitted 13 February, 2012;
originally announced February 2012.
-
Hybrid Approximate Message Passing
Authors:
Sundeep Rangan,
Alyson K. Fletcher,
Vivek K. Goyal,
Evan Byrne,
Philip Schniter
Abstract:
Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The…
▽ More
Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing interactions through aggregates of small, linearizable couplings of variables. AMP approximations based on the Central Limit Theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance--complexity trade-off can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.
△ Less
Submitted 23 March, 2017; v1 submitted 10 November, 2011;
originally announced November 2011.
-
Ranked Sparse Signal Support Detection
Authors:
Alyson K. Fletcher,
Sundeep Rangan,
Vivek K Goyal
Abstract:
This paper considers the problem of detecting the support (sparsity pattern) of a sparse vector from random noisy measurements. Conditional power of a component of the sparse vector is defined as the energy conditioned on the component being nonzero. Analysis of a simplified version of orthogonal matching pursuit (OMP) called sequential OMP (SequOMP) demonstrates the importance of knowledge of the…
▽ More
This paper considers the problem of detecting the support (sparsity pattern) of a sparse vector from random noisy measurements. Conditional power of a component of the sparse vector is defined as the energy conditioned on the component being nonzero. Analysis of a simplified version of orthogonal matching pursuit (OMP) called sequential OMP (SequOMP) demonstrates the importance of knowledge of the rankings of conditional powers. When the simple SequOMP algorithm is applied to components in nonincreasing order of conditional power, the detrimental effect of dynamic range on thresholding performance is eliminated. Furthermore, under the most favorable conditional powers, the performance of SequOMP approaches maximum likelihood performance at high signal-to-noise ratio.
△ Less
Submitted 27 October, 2011;
originally announced October 2011.
-
Estimation of Sparse MIMO Channels with Common Support
Authors:
Yann Barbotin,
Ali Hormati,
Sundeep Rangan,
Martin Vetterli
Abstract:
We consider the problem of estimating sparse communication channels in the MIMO context. In small to medium bandwidth communications, as in the current standards for OFDM and CDMA communication systems (with bandwidth up to 20 MHz), such channels are individually sparse and at the same time share a common support set. Since the underlying physical channels are inherently continuous-time, we propos…
▽ More
We consider the problem of estimating sparse communication channels in the MIMO context. In small to medium bandwidth communications, as in the current standards for OFDM and CDMA communication systems (with bandwidth up to 20 MHz), such channels are individually sparse and at the same time share a common support set. Since the underlying physical channels are inherently continuous-time, we propose a parametric sparse estimation technique based on finite rate of innovation (FRI) principles. Parametric estimation is especially relevant to MIMO communications as it allows for a robust estimation and concise description of the channels. The core of the algorithm is a generalization of conventional spectral estimation methods to multiple input signals with common support. We show the application of our technique for channel estimation in OFDM (uniformly/contiguous DFT pilots) and CDMA downlink (Walsh-Hadamard coded schemes). In the presence of additive white Gaussian noise, theoretical lower bounds on the estimation of SCS channel parameters in Rayleigh fading conditions are derived. Finally, an analytical spatial channel model is derived, and simulations on this model in the OFDM setting show the symbol error rate (SER) is reduced by a factor 2 (0 dB of SNR) to 5 (high SNR) compared to standard non-parametric methods - e.g. lowpass interpolation.
△ Less
Submitted 7 July, 2011;
originally announced July 2011.
-
Message-Passing Estimation from Quantized Samples
Authors:
Ulugbek Kamilov,
Vivek K. Goyal,
Sundeep Rangan
Abstract:
Estimation of a vector from quantized linear measurements is a common problem for which simple linear techniques are suboptimal -- sometimes greatly so. This paper develops generalized approximate message passing (GAMP) algorithms for minimum mean-squared error estimation of a random vector from quantized linear measurements, notably allowing the linear expansion to be overcomplete or undercomplet…
▽ More
Estimation of a vector from quantized linear measurements is a common problem for which simple linear techniques are suboptimal -- sometimes greatly so. This paper develops generalized approximate message passing (GAMP) algorithms for minimum mean-squared error estimation of a random vector from quantized linear measurements, notably allowing the linear expansion to be overcomplete or undercomplete and the scalar quantization to be regular or non-regular. GAMP is a recently-developed class of algorithms that uses Gaussian approximations in belief propagation and allows arbitrary separable input and output channels. Scalar quantization of measurements is incorporated into the output channel formalism, leading to the first tractable and effective method for high-dimensional estimation problems involving non-regular scalar quantization. Non-regular quantization is empirically demonstrated to greatly improve rate-distortion performance in some problems with oversampling or with undersampling combined with a sparsity-inducing prior. Under the assumption of a Gaussian measurement matrix with i.i.d. entries, the asymptotic error performance of GAMP can be accurately predicted and tracked through the state evolution formalism. We additionally use state evolution to design MSE-optimal scalar quantizers for GAMP signal reconstruction and empirically demonstrate the superior error performance of the resulting quantizers.
△ Less
Submitted 20 November, 2011; v1 submitted 31 May, 2011;
originally announced May 2011.
-
Orthogonal Matching Pursuit: A Brownian Motion Analysis
Authors:
Alyson K. Fletcher,
Sundeep Rangan
Abstract:
A well-known analysis of Tropp and Gilbert shows that orthogonal matching pursuit (OMP) can recover a k-sparse n-dimensional real vector from 4 k log(n) noise-free linear measurements obtained through a random Gaussian measurement matrix with a probability that approaches one as n approaches infinity. This work strengthens this result by showing that a lower number of measurements, 2 k log(n - k),…
▽ More
A well-known analysis of Tropp and Gilbert shows that orthogonal matching pursuit (OMP) can recover a k-sparse n-dimensional real vector from 4 k log(n) noise-free linear measurements obtained through a random Gaussian measurement matrix with a probability that approaches one as n approaches infinity. This work strengthens this result by showing that a lower number of measurements, 2 k log(n - k), is in fact sufficient for asymptotic recovery. More generally, when the sparsity level satisfies kmin <= k <= kmax but is unknown, 2 kmax log(n - kmin) measurements is sufficient. Furthermore, this number of measurements is also sufficient for detection of the sparsity pattern (support) of the vector with measurement errors provided the signal-to-noise ratio (SNR) scales to infinity. The scaling 2 k log(n - k) exactly matches the number of measurements required by the more complex lasso method for signal recovery with a similar SNR scaling.
△ Less
Submitted 29 May, 2011;
originally announced May 2011.
-
Optimal Quantization for Compressive Sensing under Message Passing Reconstruction
Authors:
Ulugbek Kamilov,
Vivek K Goyal,
Sundeep Rangan
Abstract:
We consider the optimal quantization of compressive sensing measurements following the work on generalization of relaxed belief propagation (BP) for arbitrary measurement channels. Relaxed BP is an iterative reconstruction scheme inspired by message passing algorithms on bipartite graphs. Its asymptotic error performance can be accurately predicted and tracked through the state evolution formalism…
▽ More
We consider the optimal quantization of compressive sensing measurements following the work on generalization of relaxed belief propagation (BP) for arbitrary measurement channels. Relaxed BP is an iterative reconstruction scheme inspired by message passing algorithms on bipartite graphs. Its asymptotic error performance can be accurately predicted and tracked through the state evolution formalism. We utilize these results to design mean-square optimal scalar quantizers for relaxed BP signal reconstruction and empirically demonstrate the superior error performance of the resulting quantizers.
△ Less
Submitted 13 March, 2011; v1 submitted 22 February, 2011;
originally announced February 2011.
-
Generalized Approximate Message Passing for Estimation with Random Linear Mixing
Authors:
Sundeep Rangan
Abstract:
We consider the estimation of an i.i.d.\ random vector observed through a linear transform followed by a componentwise, probabilistic (possibly nonlinear) measurement channel. A novel algorithm, called generalized approximate message passing (GAMP), is presented that provides computationally efficient approximate implementations of max-sum and sum-problem loopy belief propagation for such problems…
▽ More
We consider the estimation of an i.i.d.\ random vector observed through a linear transform followed by a componentwise, probabilistic (possibly nonlinear) measurement channel. A novel algorithm, called generalized approximate message passing (GAMP), is presented that provides computationally efficient approximate implementations of max-sum and sum-problem loopy belief propagation for such problems. The algorithm extends earlier approximate message passing methods to incorporate arbitrary distributions on both the input and output of the transform and can be applied to a wide range of problems in nonlinear compressed sensing and learning.
Extending an analysis by Bayati and Montanari, we argue that the asymptotic componentwise behavior of the GAMP method under large, i.i.d. Gaussian transforms is described by a simple set of state evolution (SE) equations. From the SE equations, one can \emph{exactly} predict the asymptotic value of virtually any componentwise performance metric including mean-squared error or detection accuracy. Moreover, the analysis is valid for arbitrary input and output distributions, even when the corresponding optimization problems are non-convex. The results match predictions by Guo and Wang for relaxed belief propagation on large sparse matrices and, in certain instances, also agree with the optimal performance predicted by the replica method. The GAMP methodology thus provides a computationally efficient methodology, applicable to a large class of non-Gaussian estimation problems with precise asymptotic performance guarantees.
△ Less
Submitted 13 August, 2012; v1 submitted 25 October, 2010;
originally announced October 2010.
-
Belief Propagation Methods for Intercell Interference Coordination
Authors:
Sundeep Rangan,
Ritesh Madan
Abstract:
We consider a broad class of interference coordination and resource allocation problems for wireless links where the goal is to maximize the sum of functions of individual link rates. Such problems arise in the context of, for example, fractional frequency reuse (FFR) for macro-cellular networks and dynamic interference management in femtocells. The resulting optimization problems are typically ha…
▽ More
We consider a broad class of interference coordination and resource allocation problems for wireless links where the goal is to maximize the sum of functions of individual link rates. Such problems arise in the context of, for example, fractional frequency reuse (FFR) for macro-cellular networks and dynamic interference management in femtocells. The resulting optimization problems are typically hard to solve optimally even using centralized algorithms but are an essential computational step in implementing rate-fair and queue stabilizing scheduling policies in wireless networks. We consider a belief propagation framework to solve such problems approximately. In particular, we construct approximations to the belief propagation iterations to obtain computationally simple and distributed algorithms with low communication overhead. Notably, our methods are very general and apply to, for example, the optimization of transmit powers, transmit beamforming vectors, and sub-band allocation to maximize the above objective. Numerical results for femtocell deployments demonstrate that such algorithms compute a very good operating point in typically just a couple of iterations.
△ Less
Submitted 31 July, 2010;
originally announced August 2010.
-
Femto-Macro Cellular Interference Control with Subband Scheduling and Interference Cancelation
Authors:
Sundeep Rangan
Abstract:
A significant technical challenge in deploying femtocells is controlling the interference from the underlay of femtos onto the overlay of macros. This paper presents a novel interference control method where the macrocell bandwidth is partitioned into subbands, and the short-range femtocell links adaptively allocate their power across the subbands based on a load-spillage power control method. The…
▽ More
A significant technical challenge in deploying femtocells is controlling the interference from the underlay of femtos onto the overlay of macros. This paper presents a novel interference control method where the macrocell bandwidth is partitioned into subbands, and the short-range femtocell links adaptively allocate their power across the subbands based on a load-spillage power control method. The scheme can improve rate distribution in the macro network while also providing opportunities for short-range communication as well. Moreover, the proposed scheme requires minimal interference coordination communication between the femtos and macros, which is one of the main challenges in femtocell systems. Also, simulations show certain advantages over simpler orthogonalization schemes or power control schemes without subband partitioning. Further modest gains may also be possible with interference cancelation.
△ Less
Submitted 11 July, 2010; v1 submitted 3 July, 2010;
originally announced July 2010.
-
Estimation with Random Linear Mixing, Belief Propagation and Compressed Sensing
Authors:
Sundeep Rangan
Abstract:
We apply Guo and Wang's relaxed belief propagation (BP) method to the estimation of a random vector from linear measurements followed by a componentwise probabilistic measurement channel. Relaxed BP uses a Gaussian approximation in standard BP to obtain significant computational savings for dense measurement matrices. The main contribution of this paper is to extend the relaxed BP method and analy…
▽ More
We apply Guo and Wang's relaxed belief propagation (BP) method to the estimation of a random vector from linear measurements followed by a componentwise probabilistic measurement channel. Relaxed BP uses a Gaussian approximation in standard BP to obtain significant computational savings for dense measurement matrices. The main contribution of this paper is to extend the relaxed BP method and analysis to general (non-AWGN) output channels. Specifically, we present detailed equations for implementing relaxed BP for general channels and show that relaxed BP has an identical asymptotic large sparse limit behavior as standard BP, as predicted by the Guo and Wang's state evolution (SE) equations. Applications are presented to compressed sensing and estimation with bounded noise.
△ Less
Submitted 17 May, 2010; v1 submitted 13 January, 2010;
originally announced January 2010.
-
Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
Authors:
Sundeep Rangan,
Alyson K. Fletcher,
Vivek K Goyal
Abstract:
The replica method is a non-rigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise…
▽ More
The replica method is a non-rigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verdu.
The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.
△ Less
Submitted 8 October, 2011; v1 submitted 17 June, 2009;
originally announced June 2009.
-
On-Off Random Access Channels: A Compressed Sensing Framework
Authors:
Alyson K. Fletcher,
Sundeep Rangan,
Vivek K Goyal
Abstract:
This paper considers a simple on-off random multiple access channel, where n users communicate simultaneously to a single receiver over m degrees of freedom. Each user transmits with probability lambda, where typically lambda n < m << n, and the receiver must detect which users transmitted. We show that when the codebook has i.i.d. Gaussian entries, detecting which users transmitted is mathemati…
▽ More
This paper considers a simple on-off random multiple access channel, where n users communicate simultaneously to a single receiver over m degrees of freedom. Each user transmits with probability lambda, where typically lambda n < m << n, and the receiver must detect which users transmitted. We show that when the codebook has i.i.d. Gaussian entries, detecting which users transmitted is mathematically equivalent to a certain sparsity detection problem considered in compressed sensing. Using recent sparsity results, we derive upper and lower bounds on the capacities of these channels. We show that common sparsity detection algorithms, such as lasso and orthogonal matching pursuit (OMP), can be used as tractable multiuser detection schemes and have significantly better performance than single-user detection. These methods do achieve some near-far resistance but--at high signal-to-noise ratios (SNRs)--may achieve capacities far below optimal maximum likelihood detection. We then present a new algorithm, called sequential OMP, that illustrates that iterative detection combined with power ordering or power sha** can significantly improve the high SNR performance. Sequential OMP is analogous to successive interference cancellation in the classic multiple access channel. Our results thereby provide insight into the roles of power control and multiuser detection on random-access signalling.
△ Less
Submitted 9 March, 2009; v1 submitted 5 March, 2009;
originally announced March 2009.
-
Necessary and Sufficient Conditions on Sparsity Pattern Recovery
Authors:
Alyson K. Fletcher,
Sundeep Rangan,
Vivek K. Goyal
Abstract:
The problem of detecting the sparsity pattern of a k-sparse vector in R^n from m random noisy measurements is of interest in many areas such as system identification, denoising, pattern recognition, and compressed sensing. This paper addresses the scaling of the number of measurements m, with signal dimension n and sparsity-level nonzeros k, for asymptotically-reliable detection. We show a neces…
▽ More
The problem of detecting the sparsity pattern of a k-sparse vector in R^n from m random noisy measurements is of interest in many areas such as system identification, denoising, pattern recognition, and compressed sensing. This paper addresses the scaling of the number of measurements m, with signal dimension n and sparsity-level nonzeros k, for asymptotically-reliable detection. We show a necessary condition for perfect recovery at any given SNR for all algorithms, regardless of complexity, is m = Omega(k log(n-k)) measurements. Conversely, it is shown that this scaling of Omega(k log(n-k)) measurements is sufficient for a remarkably simple ``maximum correlation'' estimator. Hence this scaling is optimal and does not require more sophisticated techniques such as lasso or matching pursuit. The constants for both the necessary and sufficient conditions are precisely defined in terms of the minimum-to-average ratio of the nonzero components and the SNR. The necessary condition improves upon previous results for maximum likelihood estimation. For lasso, it also provides a necessary condition at any SNR and for low SNR improves upon previous work. The sufficient condition provides the first asymptotically-reliable detection guarantee at finite SNR.
△ Less
Submitted 11 April, 2008;
originally announced April 2008.
