Showing 1–49 of 49 results for author: Goyal, V K

Resolving Multiphoton Coincidences in Single-Photon Detector Arrays with Row-Column Readouts

Authors: Shashwath Bharadwaj, Ruangrawee Kitichotkul, Akshay Agarwal, Vivek K Goyal

Abstract: Row-column multiplexing has proven to be an effective strategy in scaling single-photon detector arrays to kilopixel and megapixel spatial resolutions. However, with this readout mechanism, multiphoton coincidences on the array cannot be easily resolved due to ambiguities concerning their spatial locations of incidence. In this work, we propose a method to resolve up to 4-photon coincidences in si… ▽ More Row-column multiplexing has proven to be an effective strategy in scaling single-photon detector arrays to kilopixel and megapixel spatial resolutions. However, with this readout mechanism, multiphoton coincidences on the array cannot be easily resolved due to ambiguities concerning their spatial locations of incidence. In this work, we propose a method to resolve up to 4-photon coincidences in single-photon detector arrays with row-column readouts. By utilizing unambiguous single-photon measurements to estimate probabilities of detection at each pixel, we redistribute the ambiguous multiphoton counts among candidate pixel locations such that the peak signal-to-noise-ratio of the reconstruction is increased between 3 and 4 dB compared to conventional methods at optimal operating conditions. We also show that our method allows the operation of these arrays at higher incident photon fluxes as compared to previous methods. The application of this technique to imaging natural scenes is demonstrated using Monte Carlo experiments. △ Less

Submitted 5 December, 2023; v1 submitted 5 December, 2023; originally announced December 2023.

Comments: 13 pages, 7 figures

Shot noise-mitigated secondary electron imaging with ion count-aided microscopy

Authors: Akshay Agarwal, Leila Kasaei, Xinglin He, Ruangrawee Kitichotkul, Oguz Kagan Hitit, Minxu Peng, J. Albert Schultz, Leonard C. Feldman, Vivek K Goyal

Abstract: Modern science is dependent on imaging on the nanoscale, often achieved through processes that detect secondary electrons created by a highly focused incident charged particle beam. Multiple types of measurement noise limit the ultimate trade-off between the image quality and the incident particle dose, which can preclude useful imaging of dose-sensitive samples. Existing methods to improve image… ▽ More Modern science is dependent on imaging on the nanoscale, often achieved through processes that detect secondary electrons created by a highly focused incident charged particle beam. Multiple types of measurement noise limit the ultimate trade-off between the image quality and the incident particle dose, which can preclude useful imaging of dose-sensitive samples. Existing methods to improve image quality do not fundamentally mitigate the noise sources. Furthermore, barriers to assigning a physically meaningful scale make the images qualitative. Here we introduce ion count-aided microscopy (ICAM), which is a quantitative imaging technique that uses statistically principled estimation of the secondary electron yield. With a readily implemented change in data collection, ICAM substantially reduces source shot noise. In helium ion microscopy, we demonstrate 3x dose reduction and a good match between these empirical results and theoretical performance predictions. ICAM facilitates imaging of fragile samples and may make imaging with heavier particles more attractive. △ Less

Submitted 8 July, 2024; v1 submitted 12 November, 2023; originally announced November 2023.

Comments: Updated Introduction, Updated Discussion and Outlook, Updated Methods. Main article: 17 pages, 4 figures

Absorption-Based, Passive Range Imaging from Hyperspectral Thermal Measurements

Authors: Unay Dorken Gallastegi, Hoover Rueda-Chacon, Martin J. Stevens, Vivek K Goyal

Abstract: Passive hyperspectral long-wave infrared measurements are remarkably informative about the surroundings, such as remote object material composition, temperature, and range; and air temperature and gas concentrations. Remote object material and temperature determine the spectrum of thermal radiance, and range, air temperature, and gas concentrations determine how this spectrum is modified by propag… ▽ More Passive hyperspectral long-wave infrared measurements are remarkably informative about the surroundings, such as remote object material composition, temperature, and range; and air temperature and gas concentrations. Remote object material and temperature determine the spectrum of thermal radiance, and range, air temperature, and gas concentrations determine how this spectrum is modified by propagation to the sensor. We computationally separate these phenomena, introducing a novel passive range imaging method based on atmospheric absorption of ambient thermal radiance. Previously demonstrated passive absorption-based ranging methods assume hot and highly emitting objects. However, the temperature variation in natural scenes is usually low, making range imaging challenging. Our method benefits from explicit consideration of air emission and parametric modeling of atmospheric absorption. To mitigate noise in low-contrast scenarios, we jointly estimate range and intrinsic object properties by exploiting a variety of absorption lines spread over the infrared spectrum. Along with Monte Carlo simulations that demonstrate the importance of regularization, temperature differentials, and availability of many spectral bands, we apply this method to long-wave infrared (8--13 $μ$m) hyperspectral image data acquired from natural scenes with no active illumination. Range features from 15m to 150m are recovered, with good qualitative match to unaligned lidar data. △ Less

Submitted 10 August, 2023; originally announced August 2023.

Comments: 15 pages, 14 figures

Continuous-Time Modeling and Analysis of Particle Beam Metrology

Authors: Akshay Agarwal, Minxu Peng, Vivek K. Goyal

Abstract: Particle beam microscopy (PBM) performs nanoscale imaging by pixelwise capture of scalar values representing noisy measurements of the response from secondary electrons (SEs) integrated over a dwell time. Extended to metrology, goals include estimating SE yield at each pixel and detecting differences in SE yield across pixels; obstacles include shot noise in the particle source as well as lack of… ▽ More Particle beam microscopy (PBM) performs nanoscale imaging by pixelwise capture of scalar values representing noisy measurements of the response from secondary electrons (SEs) integrated over a dwell time. Extended to metrology, goals include estimating SE yield at each pixel and detecting differences in SE yield across pixels; obstacles include shot noise in the particle source as well as lack of knowledge of and variability in the instrument response to single SEs. A recently introduced time-resolved measurement paradigm promises mitigation of source shot noise, but its analysis and development have been largely limited to estimation problems under an idealization in which SE bursts are directly and perfectly counted. Here, analyses are extended to error exponents in feature detection problems and to degraded measurements that are representative of actual instrument behavior for estimation problems. For estimation from idealized SE counts, insights on existing estimators and a superior estimator are also provided. For estimation in a realistic PBM imaging scenario, extensions to the idealized model are introduced, methods for model parameter extraction are discussed, and large improvements from time-resolved data are presented. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Comments: 14 pages, 10 figures

Journal ref: IEEE J. Selected Areas of Information Theory, vol. 4, pp. 61-74, 9 June 2023

Denoising Particle Beam Micrographs with Plug-and-Play Methods

Authors: Minxu Peng, Ruangrawee Kitichotkul, Sheila W. Seidel, Christopher Yu, Vivek K Goyal

Abstract: In a particle beam microscope, a raster-scanned focused beam of particles interacts with a sample to generate a secondary electron (SE) signal pixel by pixel. Conventionally formed micrographs are noisy because of limitations on acquisition time and dose. Recent work has shown that estimation methods applicable to a time-resolved measurement paradigm can greatly reduce noise, but these methods app… ▽ More In a particle beam microscope, a raster-scanned focused beam of particles interacts with a sample to generate a secondary electron (SE) signal pixel by pixel. Conventionally formed micrographs are noisy because of limitations on acquisition time and dose. Recent work has shown that estimation methods applicable to a time-resolved measurement paradigm can greatly reduce noise, but these methods apply pixel by pixel without exploiting image structure. Raw SE count data can be modeled with a compound Poisson (Neyman Type A) likelihood, which implies data variance that is signal-dependent and greater than the variation in the underlying particle-sample interaction. These statistical properties make methods that assume additive white Gaussian noise ineffective. This paper introduces methods for particle beam micrograph denoising that use the plug-and-play framework to exploit image structure while being applicable to the unusual data likelihoods of this modality. Approximations of the data likelihood that vary in accuracy and computational complexity are combined with denoising by total variation regularization, BM3D, and DnCNN. Methods are provided for both conventional and time-resolved measurements, assuming SE counts are available. In simulations representative of helium ion microscopy and scanning electron microscopy, significant improvements in root mean-squared error (RMSE), structural similarity index measure (SSIM), and qualitative appearance are obtained. Average reductions in RMSE are by factors ranging from 2.24 to 4.11. △ Less

Submitted 3 May, 2023; v1 submitted 30 August, 2022; originally announced August 2022.

Journal ref: IEEE Transactions on Computational Imaging, vol. 9, pp. 581--593, 13 June 2023

Non-Line-of-Sight Tracking and Map** with an Active Corner Camera

Authors: Sheila Seidel, Hoover Rueda-Chacon, Iris Cusini, Federica Villa, Franco Zappa, Christopher Yu, Vivek K Goyal

Abstract: The ability to form non-line-of-sight (NLOS) images of changing scenes could be transformative in a variety of fields, including search and rescue, autonomous vehicle navigation, and reconnaissance. Most existing active NLOS methods illuminate the hidden scene using a pulsed laser directed at a relay surface and collect time-resolved measurements of returning light. The prevailing approaches inclu… ▽ More The ability to form non-line-of-sight (NLOS) images of changing scenes could be transformative in a variety of fields, including search and rescue, autonomous vehicle navigation, and reconnaissance. Most existing active NLOS methods illuminate the hidden scene using a pulsed laser directed at a relay surface and collect time-resolved measurements of returning light. The prevailing approaches include raster scanning of a rectangular grid on a vertical wall opposite the volume of interest to generate a collection of confocal measurements. These are inherently limited by the need for laser scanning. Methods that avoid laser scanning track the moving parts of the hidden scene as one or two point targets. In this work, based on more complete optical response modeling yet still without multiple illumination positions, we demonstrate accurate reconstructions of objects in motion and a 'map' of the stationary scenery behind them. The ability to count, localize, and characterize the sizes of hidden objects in motion, combined with map** of the stationary hidden scene, could greatly improve indoor situational awareness in a variety of applications. △ Less

Submitted 2 August, 2022; originally announced August 2022.

Online Beam Current Estimation in Particle Beam Microscopy

Authors: Sheila W. Seidel, Luisa Watkins, Minxu Peng, Akshay Agarwal, Christopher Yu, Vivek K Goyal

Abstract: In conventional particle beam microscopy, knowledge of the beam current is essential for accurate micrograph formation and sample milling. This generally necessitates offline calibration of the instrument. In this work, we establish that beam current can be estimated online, from the same secondary electron count data that is used to form micrographs. Our methods depend on the recently introduced… ▽ More In conventional particle beam microscopy, knowledge of the beam current is essential for accurate micrograph formation and sample milling. This generally necessitates offline calibration of the instrument. In this work, we establish that beam current can be estimated online, from the same secondary electron count data that is used to form micrographs. Our methods depend on the recently introduced time-resolved measurement concept, which combines multiple short measurements at a single pixel and has previously been shown to partially mitigate the effect of beam current variation on micrograph accuracy. We analyze the problem of jointly estimating beam current and secondary electron yield using the Cramer-Rao bound. Joint estimators operating at a single pixel and estimators that exploit models for inter-pixel correlation and Markov beam current variation are proposed and tested on synthetic microscopy data. Our estimates of secondary electron yield that incorporate explicit beam current estimation beat state-of-the-art methods, resulting in micrograph accuracy nearly indistinguishable from what is obtained with perfect beam current knowledge. Our novel beam current estimation could help improve milling outcomes, prevent sample damage, and enable online instrument diagnostics. △ Less

Submitted 20 November, 2021; originally announced November 2021.

Secondary Electron Count Imaging in SEM

Authors: Akshay Agarwal, John Simonaitis, Vivek K. Goyal, Karl K. Berggren

Abstract: Scanning electron microscopy (SEM) is a versatile technique used to image samples at the nanoscale. Conventional imaging by this technique relies on finding the average intensity of the signal generated on a detector by secondary electrons (SEs) emitted from the sample and is subject to noise due to variations in the voltage signal from the detector. This noise can result in degradation of the SEM… ▽ More Scanning electron microscopy (SEM) is a versatile technique used to image samples at the nanoscale. Conventional imaging by this technique relies on finding the average intensity of the signal generated on a detector by secondary electrons (SEs) emitted from the sample and is subject to noise due to variations in the voltage signal from the detector. This noise can result in degradation of the SEM image quality for a given imaging dose. SE count imaging, which uses the direct count of SEs detected from the sample instead of the average signal intensity, would overcome this limitation and lead to improvement in SEM image quality. In this paper, we implement an SE count imaging scheme by synchronously outcoupling the detector and beam scan signals from the microscope and using custom code to count detected SEs. We demonstrate a ~30% increase in the image signal-to-noise-ratio due to SE counting compared to conventional imaging. The only external hardware requirement for this imaging scheme is an oscilloscope fast enough to accurately sample the detector signal for SE counting, making the scheme easily implementable on any SEM. △ Less

Submitted 2 November, 2021; originally announced November 2021.

Time-Resolved Focused Ion Beam Microscopy: Modeling, Estimation Methods, and Analyses

Authors: Minxu Peng, John Murray-Bruce, Vivek K Goyal

Abstract: In a focused ion beam (FIB) microscope, source particles interact with a small volume of a sample to generate secondary electrons that are detected, pixel by pixel, to produce a micrograph. Randomness of the number of incident particles causes excess variation in the micrograph, beyond the variation in the underlying particle-sample interaction. We recently demonstrated that joint processing of mu… ▽ More In a focused ion beam (FIB) microscope, source particles interact with a small volume of a sample to generate secondary electrons that are detected, pixel by pixel, to produce a micrograph. Randomness of the number of incident particles causes excess variation in the micrograph, beyond the variation in the underlying particle-sample interaction. We recently demonstrated that joint processing of multiple time-resolved measurements from a single pixel can mitigate this effect of source shot noise in helium ion microscopy. This paper is focused on establishing a rigorous framework for understanding the potential for this approach. It introduces idealized continuous- and discrete-time abstractions of FIB microscopy with direct electron detection and estimation-theoretic limits of imaging performance under these measurement models. Novel estimators for use with continuous-time measurements are introduced and analyzed, and estimators for use with discrete-time measurements are analyzed and shown to approach their continuous-time counterparts as time resolution is increased. Simulated FIB microscopy results are consistent with theoretical analyses and demonstrate that substantial improvements over conventional FIB microscopy image formation are made possible by time-resolved measurement. △ Less

Submitted 18 February, 2022; v1 submitted 16 November, 2020; originally announced November 2020.

Comments: Accepted at IEEE Transactions on Computational Imaging (2021), Volume 7, Pages 547-561

Journal ref: IEEE Transactions on Computational Imaging, vol. 7, pp. 547-561, 2021

Compressively sampling the optical transmission matrix of a multimode fibre

Authors: Shuhui Li, Charles Saunders, Daniel J. Lum, John Murray-Bruce, Vivek K Goyal, Tomas Cizmar, David B. Phillips

Abstract: Measurement of the optical transmission matrix (TM) of an opaque material is an advanced form of space-variant aberration correction. Beyond imaging, TM-based methods are emerging in a range of fields including optical communications, optical micro-manipulation, and optical computing. In many cases the TM is very sensitive to perturbations in the configuration of the scattering medium it represent… ▽ More Measurement of the optical transmission matrix (TM) of an opaque material is an advanced form of space-variant aberration correction. Beyond imaging, TM-based methods are emerging in a range of fields including optical communications, optical micro-manipulation, and optical computing. In many cases the TM is very sensitive to perturbations in the configuration of the scattering medium it represents. Therefore applications often require an up-to-the-minute characterisation of the fragile TM, typically entailing hundreds to thousands of probe measurements. In this work we explore how these measurement requirements can be relaxed using the framework of compressive sensing: incorporation of prior information enables accurate estimation from fewer measurements than the dimensionality of the TM we aim to reconstruct. Examples of such priors include knowledge of a memory effect linking input and output fields, an approximate model of the optical system, or a recent but degraded TM measurement. We demonstrate this concept by reconstructing a full-size TM of a multimode fibre supporting 754 modes at compression ratios down to ~5% with good fidelity. The level of compression achievable is dependent upon the strength of our priors. We show in this case that imaging is still possible using TMs reconstructed at compression ratios down to ~1% (8 probe measurements). This compressive TM sampling strategy is quite general and may be applied to any form of scattering system about which we have some prior knowledge, including diffusers, thin layers of tissue, fibre optics of any known refractive profile, and reflections from opaque walls. These approaches offer a route to measurement of high-dimensional TMs quickly or with access to limited numbers of measurements. △ Less

Submitted 31 July, 2020; originally announced July 2020.

Comments: 17 pages, 11 figures

Two-Dimensional Non-Line-of-Sight Scene Estimation from a Single Edge Occluder

Authors: Sheila W. Seidel, John Murray-Bruce, Yanting Ma, Christopher Yu, William T. Freeman, Vivek K Goyal

Abstract: Passive non-line-of-sight imaging methods are often faster and stealthier than their active counterparts, requiring less complex and costly equipment. However, many of these methods exploit motion of an occluder or the hidden scene, or require knowledge or calibration of complicated occluders. The edge of a wall is a known and ubiquitous occluding structure that may be used as an aperture to image… ▽ More Passive non-line-of-sight imaging methods are often faster and stealthier than their active counterparts, requiring less complex and costly equipment. However, many of these methods exploit motion of an occluder or the hidden scene, or require knowledge or calibration of complicated occluders. The edge of a wall is a known and ubiquitous occluding structure that may be used as an aperture to image the region hidden behind it. Light from around the corner is cast onto the floor forming a fan-like penumbra rather than a sharp shadow. Subtle variations in the penumbra contain a remarkable amount of information about the hidden scene. Previous work has leveraged the vertical nature of the edge to demonstrate 1D (in angle measured around the corner) reconstructions of moving and stationary hidden scenery from as little as a single photograph of the penumbra. In this work, we introduce a second reconstruction dimension: range measured from the edge. We derive a new forward model, accounting for radial falloff, and propose two inversion algorithms to form 2D reconstructions from a single photograph of the penumbra. Performances of both algorithms are demonstrated on experimental data corresponding to several different hidden scene configurations. A Cramer-Rao bound analysis further demonstrates the feasibility (and utility) of the 2D corner camera. △ Less

Submitted 16 June, 2020; originally announced June 2020.

Comments: 14 pages, 15 figures

SWAGGER: Sparsity Within and Across Groups for General Estimation and Recovery

Authors: Charles Saunders, Vivek K Goyal

Abstract: Penalty functions or regularization terms that promote structured solutions to optimization problems are of great interest in many fields. Proposed in this work is a nonconvex structured sparsity penalty that promotes one-sparsity within arbitrary overlap** groups in a vector. This allows one to enforce mutual exclusivity between components within solutions to optimization problems. We show mult… ▽ More Penalty functions or regularization terms that promote structured solutions to optimization problems are of great interest in many fields. Proposed in this work is a nonconvex structured sparsity penalty that promotes one-sparsity within arbitrary overlap** groups in a vector. This allows one to enforce mutual exclusivity between components within solutions to optimization problems. We show multiple example use cases (including a total variation variant), demonstrate synergy between it and other regularizers, and propose an algorithm to efficiently solve problems regularized or constrained by the proposed penalty. △ Less

Submitted 17 June, 2020; v1 submitted 2 June, 2020; originally announced June 2020.

Comments: 7 pages, 5 figures

Seeing Around Corners with Edge-Resolved Transient Imaging

Authors: Joshua Rapp, Charles Saunders, Julián Tachella, John Murray-Bruce, Yoann Altmann, Jean-Yves Tourneret, Stephen McLaughlin, Robin M. A. Dawson, Franco N. C. Wong, Vivek K Goyal

Abstract: Non-line-of-sight (NLOS) imaging is a rapidly growing field seeking to form images of objects outside the field of view, with potential applications in search and rescue, reconnaissance, and even medical imaging. The critical challenge of NLOS imaging is that diffuse reflections scatter light in all directions, resulting in weak signals and a loss of directional information. To address this proble… ▽ More Non-line-of-sight (NLOS) imaging is a rapidly growing field seeking to form images of objects outside the field of view, with potential applications in search and rescue, reconnaissance, and even medical imaging. The critical challenge of NLOS imaging is that diffuse reflections scatter light in all directions, resulting in weak signals and a loss of directional information. To address this problem, we propose a method for seeing around corners that derives angular resolution from vertical edges and longitudinal resolution from the temporal response to a pulsed light source. We introduce an acquisition strategy, scene response model, and reconstruction algorithm that enable the formation of 2.5-dimensional representations -- a plan view plus heights -- and a 180$^{\circ}$ field of view (FOV) for large-scale scenes. Our experiments demonstrate accurate reconstructions of hidden rooms up to 3 meters in each dimension. △ Less

Submitted 17 February, 2020; originally announced February 2020.

Comments: Includes manuscript (14 pages) and supplement (24 pages)

Source Shot Noise Mitigation in Focused Ion Beam Microscopy by Time-Resolved Measurement

Authors: Minxu Peng, John Murray-Bruce, Karl K. Berggren, Vivek K Goyal

Abstract: Focused ion beam (FIB) microscopy suffers from source shot noise - random variation in the number of incident ions in any fixed dwell time - along with random variation in the number of detected secondary electrons per incident ion. This multiplicity of sources of randomness increases the variance of the measurements and thus worsens the trade-off between incident ion dose and image accuracy. Time… ▽ More Focused ion beam (FIB) microscopy suffers from source shot noise - random variation in the number of incident ions in any fixed dwell time - along with random variation in the number of detected secondary electrons per incident ion. This multiplicity of sources of randomness increases the variance of the measurements and thus worsens the trade-off between incident ion dose and image accuracy. Time-resolved sensing combined with maximum likelihood estimation from the resulting sets of measurements greatly reduces the effect of source shot noise. Through Fisher information analysis and Monte Carlo simulations, the reduction in mean-squared error or reduction in required dose is shown to be by a factor approximately equal to the secondary electron yield. Experiments with a helium ion microscope (HIM) are consistent with the analyses and suggest accuracy improvement for a fixed source dose, or reduced source dose for a desired imaging accuracy, by a factor of about 3. △ Less

Submitted 7 June, 2019; originally announced June 2019.

Beliefs in Decision-Making Cascades

Authors: Daewon Seo, Ravi Kiran Raman, Joong Bum Rhim, Vivek K Goyal, Lav R Varshney

Abstract: This work explores a social learning problem with agents having nonidentical noise variances and mismatched beliefs. We consider an $N$-agent binary hypothesis test in which each agent sequentially makes a decision based not only on a private observation, but also on preceding agents' decisions. In addition, the agents have their own beliefs instead of the true prior, and have nonidentical noise v… ▽ More This work explores a social learning problem with agents having nonidentical noise variances and mismatched beliefs. We consider an $N$-agent binary hypothesis test in which each agent sequentially makes a decision based not only on a private observation, but also on preceding agents' decisions. In addition, the agents have their own beliefs instead of the true prior, and have nonidentical noise variances in the private signal. We focus on the Bayes risk of the last agent, where preceding agents are selfish. We first derive the optimal decision rule by recursive belief update and conclude, counterintuitively, that beliefs deviating from the true prior could be optimal in this setting. The effect of nonidentical noise levels in the two-agent case is also considered and analytical properties of the optimal belief curves are given. Next, we consider a predecessor selection problem wherein the subsequent agent of a certain belief chooses a predecessor from a set of candidates with varying beliefs. We characterize the decision region for choosing such a predecessor and argue that a subsequent agent with beliefs varying from the true prior often ends up selecting a suboptimal predecessor, indicating the need for a social planner. Lastly, we discuss an augmented intelligence design problem that uses a model of human behavior from cumulative prospect theory and investigate its near-optimality and suboptimality. △ Less

Submitted 5 August, 2019; v1 submitted 23 November, 2018; originally announced December 2018.

Comments: final version, to appear in IEEE Transactions on Signal Processing

Estimation from Quantized Gaussian Measurements: When and How to Use Dither

Authors: Joshua Rapp, Robin M. A. Dawson, Vivek K Goyal

Abstract: Subtractive dither is a powerful method for removing the signal dependence of quantization noise for coarsely-quantized signals. However, estimation from dithered measurements often naively applies the sample mean or midrange, even when the total noise is not well described with a Gaussian or uniform distribution. We show that the generalized Gaussian distribution approximately describes subtracti… ▽ More Subtractive dither is a powerful method for removing the signal dependence of quantization noise for coarsely-quantized signals. However, estimation from dithered measurements often naively applies the sample mean or midrange, even when the total noise is not well described with a Gaussian or uniform distribution. We show that the generalized Gaussian distribution approximately describes subtractively-dithered, quantized samples of a Gaussian signal. Furthermore, a generalized Gaussian fit leads to simple estimators based on order statistics that match the performance of more complicated maximum likelihood estimators requiring iterative solvers. The order statistics-based estimators outperform both the sample mean and midrange for nontrivial sums of Gaussian and uniform noise. Additional analysis of the generalized Gaussian approximation yields rules of thumb for determining when and how to apply dither to quantized measurements. Specifically, we find subtractive dither to be beneficial when the ratio between the Gaussian standard deviation and quantization interval length is roughly less than 1/3. If that ratio is also greater than 0.822/$K^{0.930}$ for the number of measurements $K>20$, we present estimators more efficient than the midrange. △ Less

Submitted 22 February, 2019; v1 submitted 16 November, 2018; originally announced November 2018.

Comments: Revision with added references, figures, and appendices

Dead Time Compensation for High-Flux Ranging

Authors: Joshua Rapp, Yanting Ma, Robin M. A. Dawson, Vivek K Goyal

Abstract: Dead time effects have been considered a major limitation for fast data acquisition in various time-correlated single photon counting applications, since a commonly adopted approach for dead time mitigation is to operate in the low-flux regime where dead time effects can be ignored. Through the application of lidar ranging, this work explores the empirical distribution of detection times in the pr… ▽ More Dead time effects have been considered a major limitation for fast data acquisition in various time-correlated single photon counting applications, since a commonly adopted approach for dead time mitigation is to operate in the low-flux regime where dead time effects can be ignored. Through the application of lidar ranging, this work explores the empirical distribution of detection times in the presence of dead time and demonstrates that an accurate statistical model can result in reduced ranging error with shorter data acquisition time when operating in the high-flux regime. Specifically, we show that the empirical distribution of detection times converges to the stationary distribution of a Markov chain. Depth estimation can then be performed by passing the empirical distribution through a filter matched to the stationary distribution. Moreover, based on the Markov chain model, we formulate the recovery of arrival distribution from detection distribution as a nonlinear inverse problem and solve it via provably convergent mathematical optimization. By comparing per-detection Fisher information for depth estimation from high- and low-flux detection time distributions, we provide an analytical basis for possible improvement of ranging performance resulting from the presence of dead time. Finally, we demonstrate the effectiveness of our formulation and algorithm via simulations of lidar ranging. △ Less

Submitted 22 February, 2019; v1 submitted 25 October, 2018; originally announced October 2018.

Comments: Revision with added estimation results, references, and figures, and modified appendices

Beyond Binomial and Negative Binomial: Adaptation in Bernoulli Parameter Estimation

Authors: Safa C. Medin, John Murray-Bruce, David Castañón, Vivek K Goyal

Abstract: Estimating the parameter of a Bernoulli process arises in many applications, including photon-efficient active imaging where each illumination period is regarded as a single Bernoulli trial. Motivated by acquisition efficiency when multiple Bernoulli processes are of interest, we formulate the allocation of trials under a constraint on the mean as an optimal resource allocation problem. An oracle-… ▽ More Estimating the parameter of a Bernoulli process arises in many applications, including photon-efficient active imaging where each illumination period is regarded as a single Bernoulli trial. Motivated by acquisition efficiency when multiple Bernoulli processes are of interest, we formulate the allocation of trials under a constraint on the mean as an optimal resource allocation problem. An oracle-aided trial allocation demonstrates that there can be a significant advantage from varying the allocation for different processes and inspires a simple trial allocation gain quantity. Motivated by realizing this gain without an oracle, we present a trellis-based framework for representing and optimizing stop** rules. Considering the convenient case of Beta priors, three implementable stop** rules with similar performances are explored, and the simplest of these is shown to asymptotically achieve the oracle-aided trial allocation. These approaches are further extended to estimating functions of a Bernoulli parameter. In simulations inspired by realistic active imaging scenarios, we demonstrate significant mean-squared error improvements: up to 4.36 dB for the estimation of p and up to 1.80 dB for the estimation of log p. △ Less

Submitted 21 April, 2019; v1 submitted 24 September, 2018; originally announced September 2018.

Comments: 13 pages, 16 figures

A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging

Authors: Joshua Rapp, Vivek K Goyal

Abstract: Conventional LIDAR systems require hundreds or thousands of photon detections to form accurate depth and reflectivity images. Recent photon-efficient computational imaging methods are remarkably effective with only 1.0 to 3.0 detected photons per pixel, but they are not demonstrated at signal-to-background ratio (SBR) below 1.0 because their imaging accuracies degrade significantly in the presence… ▽ More Conventional LIDAR systems require hundreds or thousands of photon detections to form accurate depth and reflectivity images. Recent photon-efficient computational imaging methods are remarkably effective with only 1.0 to 3.0 detected photons per pixel, but they are not demonstrated at signal-to-background ratio (SBR) below 1.0 because their imaging accuracies degrade significantly in the presence of high background noise. We introduce a new approach to depth and reflectivity estimation that focuses on unmixing contributions from signal and noise sources. At each pixel in an image, short-duration range gates are adaptively determined and applied to remove detections likely to be due to noise. For pixels with too few detections to perform this censoring accurately, we borrow data from neighboring pixels to improve depth estimates, where the neighborhood formation is also adaptive to scene content. Algorithm performance is demonstrated on experimental data at varying levels of noise. Results show improved performance of both reflectivity and depth estimates over state-of-the-art methods, especially at low signal-to-background ratios. In particular, accurate imaging is demonstrated with SBR as low as 0.04. This validation of a photon-efficient, noise-tolerant method demonstrates the viability of rapid, long-range, and low-power LIDAR imaging. △ Less

Submitted 23 September, 2016; originally announced September 2016.

Journal ref: IEEE TCI vol. 3, no. 3 (2017) 445-459

Single-Photon Depth Imaging Using a Union-of-Subspaces Model

Authors: Dongeek Shin, Jeffrey H. Shapiro, Vivek K Goyal

Abstract: Light detection and ranging systems reconstruct scene depth from time-of-flight measurements. For low light-level depth imaging applications, such as remote sensing and robot vision, these systems use single-photon detectors that resolve individual photon arrivals. Even so, they must detect a large number of photons to mitigate Poisson shot noise and reject anomalous photon detections from backgro… ▽ More Light detection and ranging systems reconstruct scene depth from time-of-flight measurements. For low light-level depth imaging applications, such as remote sensing and robot vision, these systems use single-photon detectors that resolve individual photon arrivals. Even so, they must detect a large number of photons to mitigate Poisson shot noise and reject anomalous photon detections from background light. We introduce a novel framework for accurate depth imaging using a small number of detected photons in the presence of an unknown amount of background light that may vary spatially. It employs a Poisson observation model for the photon detections plus a union-of-subspaces constraint on the discrete-time flux from the scene at any single pixel. Together, they enable a greedy signal-pursuit algorithm to rapidly and simultaneously converge on accurate estimates of scene depth and background flux, without any assumptions on spatial correlations of the depth or background flux. Using experimental single-photon data, we demonstrate that our proposed framework recovers depth features with 1.7 cm absolute error, using 15 photons per image pixel and an illumination pulse with 6.7-cm scaled root-mean-square length. We also show that our framework outperforms the conventional pixelwise log-matched filtering, which is a computationally-efficient approximation to the maximum-likelihood solution, by a factor of 6.1 in absolute depth error. △ Less

Submitted 24 July, 2015; originally announced July 2015.

Comments: 5 pages

Photon-Efficient Computational 3D and Reflectivity Imaging with Single-Photon Detectors

Authors: Dongeek Shin, Ahmed Kirmani, Vivek K Goyal, Jeffrey H. Shapiro

Abstract: Capturing depth and reflectivity images at low light levels from active illumination of a scene has wide-ranging applications. Conventionally, even with single-photon detectors, hundreds of photon detections are needed at each pixel to mitigate Poisson noise. We develop a robust method for estimating depth and reflectivity using on the order of 1 detected photon per pixel averaged over the scene.… ▽ More Capturing depth and reflectivity images at low light levels from active illumination of a scene has wide-ranging applications. Conventionally, even with single-photon detectors, hundreds of photon detections are needed at each pixel to mitigate Poisson noise. We develop a robust method for estimating depth and reflectivity using on the order of 1 detected photon per pixel averaged over the scene. Our computational imager combines physically accurate single-photon counting statistics with exploitation of the spatial correlations present in real-world reflectivity and 3D structure. Experiments conducted in the presence of strong background light demonstrate that our computational imager is able to accurately recover scene depth and reflectivity, while traditional maximum-likelihood based imaging methods lead to estimates that are highly noisy. Our framework increases photon efficiency 100-fold over traditional processing and also improves, somewhat, upon first-photon imaging under a total acquisition time constraint in raster-scanned operation. Thus our new imager will be useful for rapid, low-power, and noise-tolerant active optical imaging, and its fixed dwell time will facilitate parallelization through use of a detector array. △ Less

Submitted 6 June, 2014; originally announced June 2014.

Comments: 11 pages, 8 figures

Distributed Hypothesis Testing with Social Learning and Symmetric Fusion

Authors: Joong Bum Rhim, Vivek K Goyal

Abstract: We study the utility of social learning in a distributed detection model with agents sharing the same goal: a collective decision that optimizes an agreed upon criterion. We show that social learning is helpful in some cases but is provably futile (and thus essentially a distraction) in other cases. Specifically, we consider Bayesian binary hypothesis testing performed by a distributed detection a… ▽ More We study the utility of social learning in a distributed detection model with agents sharing the same goal: a collective decision that optimizes an agreed upon criterion. We show that social learning is helpful in some cases but is provably futile (and thus essentially a distraction) in other cases. Specifically, we consider Bayesian binary hypothesis testing performed by a distributed detection and fusion system, where all decision-making agents have binary votes that carry equal weight. Decision-making agents in the team sequentially make local decisions based on their own private signals and all precedent local decisions. It is shown that the optimal decision rule is not affected by precedent local decisions when all agents observe conditionally independent and identically distributed private signals. Perfect Bayesian reasoning will cancel out all effects of social learning. When the agents observe private signals with different signal-to-noise ratios, social learning is again futile if the team decision is only approved by unanimity. Otherwise, social learning can strictly improve the team performance. Furthermore, the order in which agents make their decisions affects the team decision. △ Less

Submitted 3 April, 2014; v1 submitted 3 April, 2014; originally announced April 2014.

Comments: 10 pages, 7 figures

Journal ref: IEEE Trans. on Signal Processing, vol. 62, no. 23, pp. 6298-6308, December 2014

Social Teaching: Being Informative vs. Being Right in Sequential Decision Making

Authors: Joong Bum Rhim, Vivek K Goyal

Abstract: We show that it can be suboptimal for Bayesian decision-making agents employing social learning to use correct prior probabilities as their initial beliefs. We consider sequential Bayesian binary hypothesis testing where each individual agent makes a binary decision based on an initial belief, a private signal, and the decisions of all earlier-acting agents---with the actions of precedent agents c… ▽ More We show that it can be suboptimal for Bayesian decision-making agents employing social learning to use correct prior probabilities as their initial beliefs. We consider sequential Bayesian binary hypothesis testing where each individual agent makes a binary decision based on an initial belief, a private signal, and the decisions of all earlier-acting agents---with the actions of precedent agents causing updates of the initial belief. Each agent acts to minimize Bayes risk, with all agents sharing the same Bayes costs for Type I (false alarm) and Type II (missed detection) errors. The effect of the set of initial beliefs on the decision-making performance of the last agent is studied. The last agent makes the best decision when the initial beliefs are inaccurate. When the private signals are described by Gaussian likelihoods, the optimal initial beliefs are not haphazard but rather follow a systematic pattern: the earlier-acting agents should act as if the prior probability is larger than it is in reality when the true prior probability is small, and vice versa. We interpret this as being open minded toward the unlikely hypothesis. The early-acting agents face a trade-off between making a correct decision and being maximally informative to the later-acting agents. △ Less

Submitted 29 December, 2012; originally announced December 2012.

Keep Ballots Secret: On the Futility of Social Learning in Decision Making by Voting

Authors: Joong Bum Rhim, Vivek K. Goyal

Abstract: We show that social learning is not useful in a model of team binary decision making by voting, where each vote carries equal weight. Specifically, we consider Bayesian binary hypothesis testing where agents have any conditionally-independent observation distribution and their local decisions are fused by any L-out-of-N fusion rule. The agents make local decisions sequentially, with each allowed t… ▽ More We show that social learning is not useful in a model of team binary decision making by voting, where each vote carries equal weight. Specifically, we consider Bayesian binary hypothesis testing where agents have any conditionally-independent observation distribution and their local decisions are fused by any L-out-of-N fusion rule. The agents make local decisions sequentially, with each allowed to use its own private signal and all precedent local decisions. Though social learning generally occurs in that precedent local decisions affect an agent's belief, optimal team performance is obtained when all precedent local decisions are ignored. Thus, social learning is futile, and secret ballots are optimal. This contrasts with typical studies of social learning because we include a fusion center rather than concentrating on the performance of the latest-acting agents. △ Less

Submitted 23 December, 2012; originally announced December 2012.

Distributed Quantization Networks

Authors: John Z. Sun, Vivek K. Goyal

Abstract: Several key results in distributed source coding offer the intuition that little improvement in compression can be gained from intersensor communication when the information is coded in long blocks. However, when sensors are restricted to code their observations in small blocks (e.g., 1), intelligent collaboration between sensors can greatly reduce distortion. For networks where sensors are allowe… ▽ More Several key results in distributed source coding offer the intuition that little improvement in compression can be gained from intersensor communication when the information is coded in long blocks. However, when sensors are restricted to code their observations in small blocks (e.g., 1), intelligent collaboration between sensors can greatly reduce distortion. For networks where sensors are allowed to "chat" using a side channel that is unobservable at the fusion center, we provide asymptotically-exact characterization of distortion performance and optimal quantizer design in the high-resolution (low-distortion) regime using a framework called distributed functional scalar quantization (DFSQ). The key result is that chatting can dramatically improve performance even when intersensor communication is at very low rate, especially if the fusion center desires fidelity of a nonlinear computation applied to source realizations rather than fidelity in representing the sources themselves. We also solve the rate allocation problem when communication links have heterogeneous costs and provide a detailed example to demonstrate the theoretical and practical gains from chatting. This example for maximum computation gives insight on the gap between chatting and distributed networks, and how to optimize the intersensor communication. △ Less

Submitted 31 October, 2012; originally announced October 2012.

Journal ref: IEEE Trans. on Communications, vol. 61, no. 9, pp. 3931-3942, September 2013

Distributed Functional Scalar Quantization Simplified

Authors: John Z. Sun, Vinith Misra, Vivek K Goyal

Abstract: Distributed functional scalar quantization (DFSQ) theory provides optimality conditions and predicts performance of data acquisition systems in which a computation on acquired data is desired. We address two limitations of previous works: prohibitively expensive decoder design and a restriction to sources with bounded distributions. We rigorously show that a much simpler decoder has equivalent asy… ▽ More Distributed functional scalar quantization (DFSQ) theory provides optimality conditions and predicts performance of data acquisition systems in which a computation on acquired data is desired. We address two limitations of previous works: prohibitively expensive decoder design and a restriction to sources with bounded distributions. We rigorously show that a much simpler decoder has equivalent asymptotic performance as the conditional expectation estimator previously explored, thus reducing decoder design complexity. The simpler decoder has the feature of decoupled communication and computation blocks. Moreover, we extend the DFSQ framework with the simpler decoder to acquire sources with infinite-support distributions such as Gaussian or exponential distributions. Finally, through simulation results we demonstrate that performance at moderate coding rates is well predicted by the asymptotic analysis, and we give new insight on the rate of convergence. △ Less

Submitted 6 June, 2012; originally announced June 2012.

Journal ref: IEEE Trans. on Signal Processing, vol. 61, no. 14, pp. 3495-3508, July 2013

Hybrid Approximate Message Passing

Authors: Sundeep Rangan, Alyson K. Fletcher, Vivek K. Goyal, Evan Byrne, Philip Schniter

Abstract: Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The… ▽ More Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing interactions through aggregates of small, linearizable couplings of variables. AMP approximations based on the Central Limit Theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance--complexity trade-off can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology. △ Less

Submitted 23 March, 2017; v1 submitted 10 November, 2011; originally announced November 2011.

Ranked Sparse Signal Support Detection

Authors: Alyson K. Fletcher, Sundeep Rangan, Vivek K Goyal

Abstract: This paper considers the problem of detecting the support (sparsity pattern) of a sparse vector from random noisy measurements. Conditional power of a component of the sparse vector is defined as the energy conditioned on the component being nonzero. Analysis of a simplified version of orthogonal matching pursuit (OMP) called sequential OMP (SequOMP) demonstrates the importance of knowledge of the… ▽ More This paper considers the problem of detecting the support (sparsity pattern) of a sparse vector from random noisy measurements. Conditional power of a component of the sparse vector is defined as the energy conditioned on the component being nonzero. Analysis of a simplified version of orthogonal matching pursuit (OMP) called sequential OMP (SequOMP) demonstrates the importance of knowledge of the rankings of conditional powers. When the simple SequOMP algorithm is applied to components in nonincreasing order of conditional power, the detrimental effect of dynamic range on thresholding performance is eliminated. Furthermore, under the most favorable conditional powers, the performance of SequOMP approaches maximum likelihood performance at high signal-to-noise ratio. △ Less

Submitted 27 October, 2011; originally announced October 2011.

Comments: 13 pages

Journal ref: IEEE Trans. on Signal Processing, vol. 60, no. 11, pp. 5919-5931, November 2012

Time-Stampless Adaptive Nonuniform Sampling for Stochastic Signals

Authors: Soheil Feizi, Vivek K Goyal, Muriel Medard

Abstract: In this paper, we introduce a time-stampless adaptive nonuniform sampling (TANS) framework, in which time increments between samples are determined by a function of the $m$ most recent increments and sample values. Since only past samples are used in computing time increments, it is not necessary to save sampling times (time stamps) for use in the reconstruction process. We focus on two TANS schem… ▽ More In this paper, we introduce a time-stampless adaptive nonuniform sampling (TANS) framework, in which time increments between samples are determined by a function of the $m$ most recent increments and sample values. Since only past samples are used in computing time increments, it is not necessary to save sampling times (time stamps) for use in the reconstruction process. We focus on two TANS schemes for discrete-time stochastic signals: a greedy method, and a method based on dynamic programming. We analyze the performances of these schemes by computing (or bounding) their trade-offs between sampling rate and expected reconstruction distortion for autoregressive and Markovian signals. Simulation results support the analysis of the sampling schemes. We show that, by opportunistically adapting to local signal characteristics, TANS may lead to improved power efficiency in some applications. △ Less

Submitted 17 October, 2011; originally announced October 2011.

Comments: Submitted to IEEE Transactions on Signal Processing (partially presented at Allerton 2010)

Journal ref: IEEE Trans. on Signal Processing, vol. 60, no. 10, pp. 5440-5450, October 2012

Quantization of Prior Probabilities for Collaborative Distributed Hypothesis Testing

Authors: Joong Bum Rhim, Lav R. Varshney, Vivek K Goyal

Abstract: This paper studies the quantization of prior probabilities, drawn from an ensemble, for distributed detection and data fusion. Design and performance equivalences between a team of N agents tied by a fixed fusion rule and a more powerful single agent are obtained. Effects of identical quantization and diverse quantization are compared. Consideration of perceived common risk enables agents using di… ▽ More This paper studies the quantization of prior probabilities, drawn from an ensemble, for distributed detection and data fusion. Design and performance equivalences between a team of N agents tied by a fixed fusion rule and a more powerful single agent are obtained. Effects of identical quantization and diverse quantization are compared. Consideration of perceived common risk enables agents using diverse quantizers to collaborate in hypothesis testing, and it is proven that the minimum mean Bayes risk error is achieved by diverse quantization. The comparison shows that optimal diverse quantization with K cells per quantizer performs as well as optimal identical quantization with N(K-1)+1 cells per quantizer. Similar results are obtained for maximum Bayes risk error as the distortion criterion. △ Less

Submitted 12 September, 2011; originally announced September 2011.

Comments: 11 pages

Journal ref: IEEE Trans. on Signal Processing, vol. 60, no. 9, pp. 4537-4550, September 2012

Channels That Die

Authors: Lav R. Varshney, Sanjoy K. Mitter, Vivek K Goyal

Abstract: Given the possibility of communication systems failing catastrophically, we investigate limits to communicating over channels that fail at random times. These channels are finite-state semi-Markov channels. We show that communication with arbitrarily small probability of error is not possible. Making use of results in finite blocklength channel coding, we determine sequences of blocklengths that o… ▽ More Given the possibility of communication systems failing catastrophically, we investigate limits to communicating over channels that fail at random times. These channels are finite-state semi-Markov channels. We show that communication with arbitrarily small probability of error is not possible. Making use of results in finite blocklength channel coding, we determine sequences of blocklengths that optimize transmission volume communicated at fixed maximum message error probabilities. We provide a partial ordering of communication channels. A dynamic programming formulation is used to show the structural result that channel state feedback does not improve performance. △ Less

Submitted 10 June, 2011; originally announced June 2011.

Journal ref: IEEE Trans. on Information Theory, vol. 58, no. 9, pp. 5711-5724, September 2012

Message-Passing Estimation from Quantized Samples

Authors: Ulugbek Kamilov, Vivek K. Goyal, Sundeep Rangan

Abstract: Estimation of a vector from quantized linear measurements is a common problem for which simple linear techniques are suboptimal -- sometimes greatly so. This paper develops generalized approximate message passing (GAMP) algorithms for minimum mean-squared error estimation of a random vector from quantized linear measurements, notably allowing the linear expansion to be overcomplete or undercomplet… ▽ More Estimation of a vector from quantized linear measurements is a common problem for which simple linear techniques are suboptimal -- sometimes greatly so. This paper develops generalized approximate message passing (GAMP) algorithms for minimum mean-squared error estimation of a random vector from quantized linear measurements, notably allowing the linear expansion to be overcomplete or undercomplete and the scalar quantization to be regular or non-regular. GAMP is a recently-developed class of algorithms that uses Gaussian approximations in belief propagation and allows arbitrary separable input and output channels. Scalar quantization of measurements is incorporated into the output channel formalism, leading to the first tractable and effective method for high-dimensional estimation problems involving non-regular scalar quantization. Non-regular quantization is empirically demonstrated to greatly improve rate-distortion performance in some problems with oversampling or with undersampling combined with a sparsity-inducing prior. Under the assumption of a Gaussian measurement matrix with i.i.d. entries, the asymptotic error performance of GAMP can be accurately predicted and tracked through the state evolution formalism. We additionally use state evolution to design MSE-optimal scalar quantizers for GAMP signal reconstruction and empirically demonstrate the superior error performance of the resulting quantizers. △ Less

Submitted 20 November, 2011; v1 submitted 31 May, 2011; originally announced May 2011.

Comments: 12 pages, 8 figures

Journal ref: IEEE Trans. on Signal Processing, vol. 60, no. 12, pp. 6270-6281, December 2012

Scalar Quantization with Random Thresholds

Authors: Vivek K Goyal

Abstract: The distortion-rate performance of certain randomly-designed scalar quantizers is determined. The central results are the mean-squared error distortion and output entropy for quantizing a uniform random variable with thresholds drawn independently from a uniform distribution. The distortion is at most 6 times that of an optimal (deterministically-designed) quantizer, and for a large number of leve… ▽ More The distortion-rate performance of certain randomly-designed scalar quantizers is determined. The central results are the mean-squared error distortion and output entropy for quantizing a uniform random variable with thresholds drawn independently from a uniform distribution. The distortion is at most 6 times that of an optimal (deterministically-designed) quantizer, and for a large number of levels the output entropy is reduced by approximately (1-gamma)/(ln 2) bits, where gamma is the Euler-Mascheroni constant. This shows that the high-rate asymptotic distortion of these quantizers in an entropy-constrained context is worse than the optimal quantizer by at most a factor of 6 exp(-2(1-gamma)). △ Less

Submitted 5 July, 2011; v1 submitted 10 May, 2011; originally announced May 2011.

Comments: 4 pages, 2 figures

Journal ref: IEEE Signal Processing Letters, vol. 18, no. 9, pp. 525-528, September 2011

Optimal Quantization for Compressive Sensing under Message Passing Reconstruction

Authors: Ulugbek Kamilov, Vivek K Goyal, Sundeep Rangan

Abstract: We consider the optimal quantization of compressive sensing measurements following the work on generalization of relaxed belief propagation (BP) for arbitrary measurement channels. Relaxed BP is an iterative reconstruction scheme inspired by message passing algorithms on bipartite graphs. Its asymptotic error performance can be accurately predicted and tracked through the state evolution formalism… ▽ More We consider the optimal quantization of compressive sensing measurements following the work on generalization of relaxed belief propagation (BP) for arbitrary measurement channels. Relaxed BP is an iterative reconstruction scheme inspired by message passing algorithms on bipartite graphs. Its asymptotic error performance can be accurately predicted and tracked through the state evolution formalism. We utilize these results to design mean-square optimal scalar quantizers for relaxed BP signal reconstruction and empirically demonstrate the superior error performance of the resulting quantizers. △ Less

Submitted 13 March, 2011; v1 submitted 22 February, 2011; originally announced February 2011.

Comments: 5 pages, 3 figures, submitted to IEEE International Symposium on Information Theory (ISIT) 2011; minor corrections in v2

Bayesian Post-Processing Methods for Jitter Mitigation in Sampling

Authors: Daniel S. Weller, Vivek K Goyal

Abstract: Minimum mean squared error (MMSE) estimators of signals from samples corrupted by jitter (timing noise) and additive noise are nonlinear, even when the signal prior and additive noise have normal distributions. This paper develops a stochastic algorithm based on Gibbs sampling and slice sampling to approximate the optimal MMSE estimator in this Bayesian formulation. Simulations demonstrate that th… ▽ More Minimum mean squared error (MMSE) estimators of signals from samples corrupted by jitter (timing noise) and additive noise are nonlinear, even when the signal prior and additive noise have normal distributions. This paper develops a stochastic algorithm based on Gibbs sampling and slice sampling to approximate the optimal MMSE estimator in this Bayesian formulation. Simulations demonstrate that this nonlinear algorithm can improve significantly upon the linear MMSE estimator, as well as the EM algorithm approximation to the maximum likelihood (ML) estimator used in classical estimation. Effective off-chip post-processing to mitigate jitter enables greater jitter to be tolerated, potentially reducing on-chip ADC power consumption. △ Less

Submitted 8 October, 2010; v1 submitted 28 July, 2010; originally announced July 2010.

Comments: 12 pages, 11 figures

Journal ref: IEEE Trans. on Signal Processing, vol. 59, no. 5, pp. 2112-2123, May 2011

On the Estimation of Nonrandom Signal Coefficients from Jittered Samples

Authors: Daniel S. Weller, Vivek K Goyal

Abstract: This paper examines the problem of estimating the parameters of a bandlimited signal from samples corrupted by random jitter (timing noise) and additive iid Gaussian noise, where the signal lies in the span of a finite basis. For the presented classical estimation problem, the Cramer-Rao lower bound (CRB) is computed, and an Expectation-Maximization (EM) algorithm approximating the maximum likelih… ▽ More This paper examines the problem of estimating the parameters of a bandlimited signal from samples corrupted by random jitter (timing noise) and additive iid Gaussian noise, where the signal lies in the span of a finite basis. For the presented classical estimation problem, the Cramer-Rao lower bound (CRB) is computed, and an Expectation-Maximization (EM) algorithm approximating the maximum likelihood (ML) estimator is developed. Simulations are performed to study the convergence properties of the EM algorithm and compare the performance both against the CRB and a basic linear estimator. These simulations demonstrate that by post-processing the jittered samples with the proposed EM algorithm, greater jitter can be tolerated, potentially reducing on-chip ADC power consumption substantially. △ Less

Submitted 28 July, 2010; originally announced July 2010.

Comments: 11 pages, 8 figures

Journal ref: IEEE Trans. on Signal Processing, vol. 59, no. 2, pp. 587-597, February 2011

Frame Permutation Quantization

Authors: Ha Q. Nguyen, Vivek K Goyal, Lav R. Varshney

Abstract: Frame permutation quantization (FPQ) is a new vector quantization technique using finite frames. In FPQ, a vector is encoded using a permutation source code to quantize its frame expansion. This means that the encoding is a partial ordering of the frame expansion coefficients. Compared to ordinary permutation source coding, FPQ produces a greater number of possible quantization rates and a higher… ▽ More Frame permutation quantization (FPQ) is a new vector quantization technique using finite frames. In FPQ, a vector is encoded using a permutation source code to quantize its frame expansion. This means that the encoding is a partial ordering of the frame expansion coefficients. Compared to ordinary permutation source coding, FPQ produces a greater number of possible quantization rates and a higher maximum rate. Various representations for the partitions induced by FPQ are presented, and reconstruction algorithms based on linear programming, quadratic programming, and recursive orthogonal projection are derived. Implementations of the linear and quadratic programming algorithms for uniform and Gaussian sources show performance improvements over entropy-constrained scalar quantization for certain combinations of vector dimension and coding rate. Monte Carlo evaluation of the recursive algorithm shows that mean-squared error (MSE) decays as 1/M^4 for an M-element frame, which is consistent with previous results on optimal decay of MSE. Reconstruction using the canonical dual frame is also studied, and several results relate properties of the analysis frame to whether linear reconstruction techniques provide consistent reconstructions. △ Less

Submitted 22 November, 2010; v1 submitted 8 September, 2009; originally announced September 2009.

Comments: 29 pages, 5 figures; detailed added to proof of Theorem 4.3 and a few minor corrections

Journal ref: Applied and Computational Harmonic Analysis, vol 31, no, 1, pp,. 74-97, July 2011

Concentric Permutation Source Codes

Authors: Ha Q. Nguyen, Lav R. Varshney, Vivek K Goyal

Abstract: Permutation codes are a class of structured vector quantizers with a computationally-simple encoding procedure based on sorting the scalar components. Using a codebook comprising several permutation codes as subcodes preserves the simplicity of encoding while increasing the number of rate-distortion operating points, improving the convex hull of operating points, and increasing design complexity.… ▽ More Permutation codes are a class of structured vector quantizers with a computationally-simple encoding procedure based on sorting the scalar components. Using a codebook comprising several permutation codes as subcodes preserves the simplicity of encoding while increasing the number of rate-distortion operating points, improving the convex hull of operating points, and increasing design complexity. We show that when the subcodes are designed with the same composition, optimization of the codebook reduces to a lower-dimensional vector quantizer design within a single cone. Heuristics for reducing design complexity are presented, including an optimization of the rate allocation in a shape-gain vector quantizer with gain-dependent wrapped spherical shape codebook. △ Less

Submitted 23 June, 2010; v1 submitted 3 September, 2009; originally announced September 2009.

Journal ref: IEEE Transactions on Communications, vol. 58, no. 11, pp. 3154-3164, November 2010

Simultaneously Sparse Solutions to Linear Inverse Problems with Multiple System Matrices and a Single Observation Vector

Authors: Adam C. Zelinski, Vivek K Goyal, Elfar Adalsteinsson

Abstract: A linear inverse problem is proposed that requires the determination of multiple unknown signal vectors. Each unknown vector passes through a different system matrix and the results are added to yield a single observation vector. Given the matrices and lone observation, the objective is to find a simultaneously sparse set of unknown vectors that solves the system. We will refer to this as the mu… ▽ More A linear inverse problem is proposed that requires the determination of multiple unknown signal vectors. Each unknown vector passes through a different system matrix and the results are added to yield a single observation vector. Given the matrices and lone observation, the objective is to find a simultaneously sparse set of unknown vectors that solves the system. We will refer to this as the multiple-system single-output (MSSO) simultaneous sparsity problem. This manuscript contrasts the MSSO problem with other simultaneous sparsity problems and conducts a thorough initial exploration of algorithms with which to solve it. Seven algorithms are formulated that approximately solve this NP-Hard problem. Three greedy techniques are developed (matching pursuit, orthogonal matching pursuit, and least squares matching pursuit) along with four methods based on a convex relaxation (iteratively reweighted least squares, two forms of iterative shrinkage, and formulation as a second-order cone program). The algorithms are evaluated across three experiments: the first and second involve sparsity profile recovery in noiseless and noisy scenarios, respectively, while the third deals with magnetic resonance imaging radio-frequency excitation pulse design. △ Less

Submitted 12 July, 2009; originally announced July 2009.

Comments: 36 pages; manuscript unchanged from July 21, 2008, except for updated references; content appears in September 2008 PhD thesis

Journal ref: SIAM J. Scientific Computing, vol. 31, no. 5, pp. 4533-4579, January 2010

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

Authors: Sundeep Rangan, Alyson K. Fletcher, Vivek K Goyal

Abstract: The replica method is a non-rigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise… ▽ More The replica method is a non-rigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verdu. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability. △ Less

Submitted 8 October, 2011; v1 submitted 17 June, 2009; originally announced June 2009.

Comments: 22 pages; added details on the replica symmetry assumption

Journal ref: IEEE Trans. on Information Theory, vol. 58, no. 3, pp. 1903-1923, March 2012

On-Off Random Access Channels: A Compressed Sensing Framework

Authors: Alyson K. Fletcher, Sundeep Rangan, Vivek K Goyal

Abstract: This paper considers a simple on-off random multiple access channel, where n users communicate simultaneously to a single receiver over m degrees of freedom. Each user transmits with probability lambda, where typically lambda n < m << n, and the receiver must detect which users transmitted. We show that when the codebook has i.i.d. Gaussian entries, detecting which users transmitted is mathemati… ▽ More This paper considers a simple on-off random multiple access channel, where n users communicate simultaneously to a single receiver over m degrees of freedom. Each user transmits with probability lambda, where typically lambda n < m << n, and the receiver must detect which users transmitted. We show that when the codebook has i.i.d. Gaussian entries, detecting which users transmitted is mathematically equivalent to a certain sparsity detection problem considered in compressed sensing. Using recent sparsity results, we derive upper and lower bounds on the capacities of these channels. We show that common sparsity detection algorithms, such as lasso and orthogonal matching pursuit (OMP), can be used as tractable multiuser detection schemes and have significantly better performance than single-user detection. These methods do achieve some near-far resistance but--at high signal-to-noise ratios (SNRs)--may achieve capacities far below optimal maximum likelihood detection. We then present a new algorithm, called sequential OMP, that illustrates that iterative detection combined with power ordering or power sha** can significantly improve the high SNR performance. Sequential OMP is analogous to successive interference cancellation in the classic multiple access channel. Our results thereby provide insight into the roles of power control and multiuser detection on random-access signalling. △ Less

Submitted 9 March, 2009; v1 submitted 5 March, 2009; originally announced March 2009.

Comments: 18 pages, 5 figures; addition of inadvertently omitted support information and acknowledgments

Distributed Scalar Quantization for Computing: High-Resolution Analysis and Extensions

Authors: Vinith Misra, Vivek K Goyal, Lav R. Varshney

Abstract: Communication of quantized information is frequently followed by a computation. We consider situations of \emph{distributed functional scalar quantization}: distributed scalar quantization of (possibly correlated) sources followed by centralized computation of a function. Under smoothness conditions on the sources and function, companding scalar quantizer designs are developed to minimize mean-squ… ▽ More Communication of quantized information is frequently followed by a computation. We consider situations of \emph{distributed functional scalar quantization}: distributed scalar quantization of (possibly correlated) sources followed by centralized computation of a function. Under smoothness conditions on the sources and function, companding scalar quantizer designs are developed to minimize mean-squared error (MSE) of the computed function as the quantizer resolution is allowed to grow. Striking improvements over quantizers designed without consideration of the function are possible and are larger in the entropy-constrained setting than in the fixed-rate setting. As extensions to the basic analysis, we characterize a large class of functions for which regular quantization suffices, consider certain functions for which asymptotic optimality is achieved without arbitrarily fine quantization, and allow limited collaboration between source encoders. In the entropy-constrained setting, a single bit per sample communicated between encoders can have an arbitrarily-large effect on functional distortion. In contrast, such communication has very little effect in the fixed-rate setting. △ Less

Submitted 12 May, 2011; v1 submitted 21 November, 2008; originally announced November 2008.

Comments: 36 pages, 10 figures

Journal ref: IEEE Trans. on Information Theory, vol. 57, no. 8, pp. 5298-5325, August 2011

Nonlinear Digital Post-Processing to Mitigate Jitter in Sampling

Authors: Daniel S. Weller, Vivek K Goyal

Abstract: This paper describes several new algorithms for estimating the parameters of a periodic bandlimited signal from samples corrupted by jitter (timing noise) and additive noise. Both classical (non-random) and Bayesian formulations are considered: an Expectation-Maximization (EM) algorithm is developed to compute the maximum likelihood (ML) estimator for the classical estimation framework, and two… ▽ More This paper describes several new algorithms for estimating the parameters of a periodic bandlimited signal from samples corrupted by jitter (timing noise) and additive noise. Both classical (non-random) and Bayesian formulations are considered: an Expectation-Maximization (EM) algorithm is developed to compute the maximum likelihood (ML) estimator for the classical estimation framework, and two Gibbs samplers are proposed to approximate the Bayes least squares (BLS) estimate for parameters independently distributed according to a uniform prior. Simulations are performed to demonstrate the significant performance improvement achievable using these algorithms as compared to linear estimators. The ML estimator is also compared to the Cramer-Rao lower bound to determine the range of jitter for which the estimator is approximately efficient. These simulations provide evidence that the nonlinear algorithms derived here can tolerate 1.4-2 times more jitter than linear estimators, reducing on-chip ADC power consumption by 50-75 percent. △ Less

Submitted 24 September, 2008; originally announced September 2008.

Comments: 24 pages, 8 figures

Malleable Coding with Fixed Reuse

Authors: Lav R. Varshney, Julius Kusuma, Vivek K Goyal

Abstract: In cloud computing, storage area networks, remote backup storage, and similar settings, stored data is modified with updates from new versions. Representing information and modifying the representation are both expensive. Therefore it is desirable for the data to not only be compressed but to also be easily modified during updates. A malleable coding scheme considers both compression efficiency an… ▽ More In cloud computing, storage area networks, remote backup storage, and similar settings, stored data is modified with updates from new versions. Representing information and modifying the representation are both expensive. Therefore it is desirable for the data to not only be compressed but to also be easily modified during updates. A malleable coding scheme considers both compression efficiency and ease of alteration, promoting codeword reuse. We examine the trade-off between compression efficiency and malleability cost-the difficulty of synchronizing compressed versions-measured as the length of a reused prefix portion. Through a coding theorem, the region of achievable rates and malleability is expressed as a single-letter optimization. Relationships to common information problems are also described. △ Less

Submitted 9 May, 2011; v1 submitted 3 September, 2008; originally announced September 2008.

Malleable Coding: Compressed Palimpsests

Authors: Lav R. Varshney, Julius Kusuma, Vivek K Goyal

Abstract: A malleable coding scheme considers not only compression efficiency but also the ease of alteration, thus encouraging some form of recycling of an old compressed version in the formation of a new one. Malleability cost is the difficulty of synchronizing compressed versions, and malleable codes are of particular interest when representing information and modifying the representation are both expe… ▽ More A malleable coding scheme considers not only compression efficiency but also the ease of alteration, thus encouraging some form of recycling of an old compressed version in the formation of a new one. Malleability cost is the difficulty of synchronizing compressed versions, and malleable codes are of particular interest when representing information and modifying the representation are both expensive. We examine the trade-off between compression efficiency and malleability cost under a malleability metric defined with respect to a string edit distance. This problem introduces a metric topology to the compressed domain. We characterize the achievable rates and malleability as the solution of a subgraph isomorphism problem. This can be used to argue that allowing conditional entropy of the edited message given the original message to grow linearly with block length creates an exponential increase in code length. △ Less

Submitted 28 June, 2008; originally announced June 2008.

Comments: 39 pages

Necessary and Sufficient Conditions on Sparsity Pattern Recovery

Authors: Alyson K. Fletcher, Sundeep Rangan, Vivek K. Goyal

Abstract: The problem of detecting the sparsity pattern of a k-sparse vector in R^n from m random noisy measurements is of interest in many areas such as system identification, denoising, pattern recognition, and compressed sensing. This paper addresses the scaling of the number of measurements m, with signal dimension n and sparsity-level nonzeros k, for asymptotically-reliable detection. We show a neces… ▽ More The problem of detecting the sparsity pattern of a k-sparse vector in R^n from m random noisy measurements is of interest in many areas such as system identification, denoising, pattern recognition, and compressed sensing. This paper addresses the scaling of the number of measurements m, with signal dimension n and sparsity-level nonzeros k, for asymptotically-reliable detection. We show a necessary condition for perfect recovery at any given SNR for all algorithms, regardless of complexity, is m = Omega(k log(n-k)) measurements. Conversely, it is shown that this scaling of Omega(k log(n-k)) measurements is sufficient for a remarkably simple ``maximum correlation'' estimator. Hence this scaling is optimal and does not require more sophisticated techniques such as lasso or matching pursuit. The constants for both the necessary and sufficient conditions are precisely defined in terms of the minimum-to-average ratio of the nonzero components and the SNR. The necessary condition improves upon previous results for maximum likelihood estimation. For lasso, it also provides a necessary condition at any SNR and for low SNR improves upon previous work. The sufficient condition provides the first asymptotically-reliable detection guarantee at finite SNR. △ Less

Submitted 11 April, 2008; originally announced April 2008.

Comments: Submitted to IEEE Transactions on Information Theory

Journal ref: IEEE Trans. on Information Theory, vol. 55, no. 12, pp. 5758-5772, December 2009

Beyond Thresholding: Analysis and Improvements for Deterministic Parameter Estimation

Authors: Baris I. Erkmen, Vivek K. Goyal

Abstract: Hard-threshold estimators are popular in signal processing applications. We provide a detailed study of using hard-threshold estimators for estimating an unknown deterministic signal when additive white Gaussian noise corrupts observations. The analysis, depending heavily on Cram{é}r-Rao bounds, motivates piecewise-linear estimation as a simple improvement to hard thresholding. We compare the pe… ▽ More Hard-threshold estimators are popular in signal processing applications. We provide a detailed study of using hard-threshold estimators for estimating an unknown deterministic signal when additive white Gaussian noise corrupts observations. The analysis, depending heavily on Cram{é}r-Rao bounds, motivates piecewise-linear estimation as a simple improvement to hard thresholding. We compare the performance of two piecewise-linear estimators to a hard-threshold estimator. When either piecewise-linear estimator is optimized for the decay rate of the basis coefficients, its performance is better than the best possible with hard thresholding. △ Less

Submitted 22 January, 2008; originally announced January 2008.

Comments: 18 pages, 11 figures

Estimating Signals with Finite Rate of Innovation from Noisy Samples: A Stochastic Algorithm

Authors: Vincent Y. F. Tan, Vivek K. Goyal

Abstract: As an example of the recently-introduced concept of rate of innovation, signals that are linear combinations of a finite number of Diracs per unit time can be acquired by linear filtering followed by uniform sampling. However, in reality, samples are rarely noiseless. In this paper, we introduce a novel stochastic algorithm to reconstruct a signal with finite rate of innovation from its noisy sa… ▽ More As an example of the recently-introduced concept of rate of innovation, signals that are linear combinations of a finite number of Diracs per unit time can be acquired by linear filtering followed by uniform sampling. However, in reality, samples are rarely noiseless. In this paper, we introduce a novel stochastic algorithm to reconstruct a signal with finite rate of innovation from its noisy samples. Even though variants of this problem has been approached previously, satisfactory solutions are only available for certain classes of sampling kernels, for example kernels which satisfy the Strang-Fix condition. In this paper, we consider the infinite-support Gaussian kernel, which does not satisfy the Strang-Fix condition. Other classes of kernels can be employed. Our algorithm is based on Gibbs sampling, a Markov chain Monte Carlo (MCMC) method. Extensive numerical simulations demonstrate the accuracy and robustness of our algorithm. △ Less

Submitted 2 January, 2008; v1 submitted 1 January, 2008; originally announced January 2008.

Comments: Submitted to IEEE Transactions on Signal Processing

Journal ref: IEEE Trans. on Signal Processing, vol. 56, no. 10, pp. 5135-5146, October 2008

Benefiting from Disorder: Source Coding for Unordered Data

Authors: Lav R. Varshney, Vivek K. Goyal

Abstract: The order of letters is not always relevant in a communication task. This paper discusses the implications of order irrelevance on source coding, presenting results in several major branches of source coding theory: lossless coding, universal lossless coding, rate-distortion, high-rate quantization, and universal lossy coding. The main conclusions demonstrate that there is a significant rate sav… ▽ More The order of letters is not always relevant in a communication task. This paper discusses the implications of order irrelevance on source coding, presenting results in several major branches of source coding theory: lossless coding, universal lossless coding, rate-distortion, high-rate quantization, and universal lossy coding. The main conclusions demonstrate that there is a significant rate savings when order is irrelevant. In particular, lossless coding of n letters from a finite alphabet requires Theta(log n) bits and universal lossless coding requires n + o(n) bits for many countable alphabet sources. However, there are no universal schemes that can drive a strong redundancy measure to zero. Results for lossy coding include distribution-free expressions for the rate savings from order irrelevance in various high-rate quantization schemes. Rate-distortion bounds are given, and it is shown that the analogue of the Shannon lower bound is loose at all finite rates. △ Less

Submitted 16 August, 2007; originally announced August 2007.

Comments: 35 pages