-
Physics-Informed Koopman Network
Authors:
Yuying Liu,
Aleksei Sholokhov,
Hassan Mansour,
Saleh Nabi
Abstract:
Koopman operator theory is receiving increased attention due to its promise to linearize nonlinear dynamics. Neural networks that are developed to represent Koopman operators have shown great success thanks to their ability to approximate arbitrarily complex functions. However, despite their great potential, they typically require large training data-sets either from measurements of a real system…
▽ More
Koopman operator theory is receiving increased attention due to its promise to linearize nonlinear dynamics. Neural networks that are developed to represent Koopman operators have shown great success thanks to their ability to approximate arbitrarily complex functions. However, despite their great potential, they typically require large training data-sets either from measurements of a real system or from high-fidelity simulations. In this work, we propose a novel architecture inspired by physics-informed neural networks, which leverage automatic differentiation to impose the underlying physical laws via soft penalty constraints during model training. We demonstrate that it not only reduces the need of large training data-sets, but also maintains high effectiveness in approximating Koopman eigenfunctions.
△ Less
Submitted 17 November, 2022;
originally announced November 2022.
-
Fast and High-Quality Blind Multi-Spectral Image Pansharpening
Authors:
Lantao Yu,
Dehong Liu,
Hassan Mansour,
Petros T. Boufounos
Abstract:
Blind pansharpening addresses the problem of generating a high spatial-resolution multi-spectral (HRMS) image given a low spatial-resolution multi-spectral (LRMS) image with the guidance of its associated spatially misaligned high spatial-resolution panchromatic (PAN) image without parametric side information. In this paper, we propose a fast approach to blind pansharpening and achieve state-of-th…
▽ More
Blind pansharpening addresses the problem of generating a high spatial-resolution multi-spectral (HRMS) image given a low spatial-resolution multi-spectral (LRMS) image with the guidance of its associated spatially misaligned high spatial-resolution panchromatic (PAN) image without parametric side information. In this paper, we propose a fast approach to blind pansharpening and achieve state-of-the-art image reconstruction quality. Typical blind pansharpening algorithms are often computationally intensive since the blur kernel and the target HRMS image are often computed using iterative solvers and in an alternating fashion. To achieve fast blind pansharpening, we decouple the solution of the blur kernel and of the HRMS image. First, we estimate the blur kernel by computing the kernel coefficients with minimum total generalized variation that blur a downsampled version of the PAN image to approximate a linear combination of the LRMS image channels. Then, we estimate each channel of the HRMS image using local Laplacian prior to regularize the relationship between each HRMS channel and the PAN image. Solving the HRMS image is accelerated by both parallelizing across the channels and by fast numerical algorithms for each channel. Due to the fast scheme and the powerful priors we used on the blur kernel coefficients (total generalized variation) and on the cross-channel relationship (local Laplacian prior), numerical experiments demonstrate that our algorithm outperforms state-of-the-art model-based counterparts in terms of both computational time and reconstruction quality of the HRMS images.
△ Less
Submitted 30 July, 2021; v1 submitted 17 March, 2021;
originally announced March 2021.
-
Multiview Sensing With Unknown Permutations: An Optimal Transport Approach
Authors:
Yanting Ma,
Petros T. Boufounos,
Hassan Mansour,
Shuchin Aeron
Abstract:
In several applications, including imaging of deformable objects while in motion, simultaneous localization and map**, and unlabeled sensing, we encounter the problem of recovering a signal that is measured subject to unknown permutations. In this paper we take a fresh look at this problem through the lens of optimal transport (OT). In particular, we recognize that in most practical applications…
▽ More
In several applications, including imaging of deformable objects while in motion, simultaneous localization and map**, and unlabeled sensing, we encounter the problem of recovering a signal that is measured subject to unknown permutations. In this paper we take a fresh look at this problem through the lens of optimal transport (OT). In particular, we recognize that in most practical applications the unknown permutations are not arbitrary but some are more likely to occur than others. We exploit this by introducing a regularization function that promotes the more likely permutations in the solution. We show that, even though the general problem is not convex, an appropriate relaxation of the resulting regularized problem allows us to exploit the well-developed machinery of OT and develop a tractable algorithm.
△ Less
Submitted 12 March, 2021;
originally announced March 2021.
-
High-Contrast Reflection Tomography with Total-Variation Constraints
Authors:
A**kya Kadu,
Hassan Mansour,
Petros T. Boufounos
Abstract:
Inverse scattering is the process of estimating the spatial distribution of the scattering potential of an object by measuring the scattered wavefields around it. In this paper, we consider reflection tomography of high contrast objects that commonly occurs in ground-penetrating radar, exploration geophysics, terahertz imaging, ultrasound, and electron microscopy. Unlike conventional transmission…
▽ More
Inverse scattering is the process of estimating the spatial distribution of the scattering potential of an object by measuring the scattered wavefields around it. In this paper, we consider reflection tomography of high contrast objects that commonly occurs in ground-penetrating radar, exploration geophysics, terahertz imaging, ultrasound, and electron microscopy. Unlike conventional transmission tomography, the reflection regime is severely ill-posed since the measured wavefields contain far less spatial frequency information of the target object. We propose a constrained incremental frequency inversion framework that requires no side information from a background model of the object. Our framework solves a sequence of regularized least-squares subproblems that ensure consistency with the measured scattered wavefield while imposing total-variation and non-negativity constraints. We propose a proximal Quasi-Newton method to solve the resulting subproblem and devise an automatic parameter selection routine to determine the constraint of each subproblem. We validate the performance of our approach on synthetic low-resolution phantoms and with a mismatched forward model test on a high-resolution phantom.
△ Less
Submitted 14 December, 2020; v1 submitted 6 May, 2020;
originally announced May 2020.
-
On the modes of convergence of Stochastic Optimistic Mirror Descent (OMD) for saddle point problems
Authors:
Yanting Ma,
Shuchin Aeron,
Hassan Mansour
Abstract:
In this article, we study the convergence of Mirror Descent (MD) and Optimistic Mirror Descent (OMD) for saddle point problems satisfying the notion of coherence as proposed in Mertikopoulos et al. We prove convergence of OMD with exact gradients for coherent saddle point problems, and show that monotone convergence only occurs after some sufficiently large number of iterations. This is in contras…
▽ More
In this article, we study the convergence of Mirror Descent (MD) and Optimistic Mirror Descent (OMD) for saddle point problems satisfying the notion of coherence as proposed in Mertikopoulos et al. We prove convergence of OMD with exact gradients for coherent saddle point problems, and show that monotone convergence only occurs after some sufficiently large number of iterations. This is in contrast to the claim in Mertikopoulos et al. of monotone convergence of OMD with exact gradients for coherent saddle point problems. Besides highlighting this important subtlety, we note that the almost sure convergence guarantees of MD and OMD with stochastic gradients for strictly coherent saddle point problems that are claimed in Mertikopoulos et al. are not fully justified by their proof. As such, we fill out the missing details in the proof and as a result have only been able to prove convergence with high probability.
We would like to note that our analysis relies heavily on the core ideas and proof techniques introduced in Zhou et al. and Mertikopoulos et al., and we only aim to re-state and correct the results in light of what we were able to prove rigorously while filling in the much needed missing details in their proofs.
△ Less
Submitted 2 August, 2019;
originally announced August 2019.
-
CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation
Authors:
Jiawei Ma,
Zheng Shou,
Alireza Zareian,
Hassan Mansour,
Anthony Vetro,
Shih-Fu Chang
Abstract:
Many real-world applications involve multivariate, geo-tagged time series data: at each location, multiple sensors record corresponding measurements. For example, air quality monitoring system records PM2.5, CO, etc. The resulting time-series data often has missing values due to device outages or communication errors. In order to impute the missing values, state-of-the-art methods are built on Rec…
▽ More
Many real-world applications involve multivariate, geo-tagged time series data: at each location, multiple sensors record corresponding measurements. For example, air quality monitoring system records PM2.5, CO, etc. The resulting time-series data often has missing values due to device outages or communication errors. In order to impute the missing values, state-of-the-art methods are built on Recurrent Neural Networks (RNN), which process each time stamp sequentially, prohibiting the direct modeling of the relationship between distant time stamps. Recently, the self-attention mechanism has been proposed for sequence modeling tasks such as machine translation, significantly outperforming RNN because the relationship between each two time stamps can be modeled explicitly. In this paper, we are the first to adapt the self-attention mechanism for multivariate, geo-tagged time series data. In order to jointly capture the self-attention across multiple dimensions, including time, location and the sensor measurements, while maintain low computational complexity, we propose a novel approach called Cross-Dimensional Self-Attention (CDSA) to process each dimension sequentially, yet in an order-independent manner. Our extensive experiments on four real-world datasets, including three standard benchmarks and our newly collected NYC-traffic dataset, demonstrate that our approach outperforms the state-of-the-art imputation and forecasting methods. A detailed systematic analysis confirms the effectiveness of our design choices.
△ Less
Submitted 5 August, 2019; v1 submitted 23 May, 2019;
originally announced May 2019.
-
Recovery Analysis of Damped Spectrally Sparse Signals and Its Relation to MUSIC
Authors:
Shuang Li,
Hassan Mansour,
Michael B. Wakin
Abstract:
One of the classical approaches for estimating the frequencies and dam** factors in a spectrally sparse signal is the MUSIC algorithm, which exploits the low-rank structure of an autocorrelation matrix. Low-rank matrices have also received considerable attention recently in the context of optimization algorithms with partial observations, and nuclear norm minimization (NNM) has been widely used…
▽ More
One of the classical approaches for estimating the frequencies and dam** factors in a spectrally sparse signal is the MUSIC algorithm, which exploits the low-rank structure of an autocorrelation matrix. Low-rank matrices have also received considerable attention recently in the context of optimization algorithms with partial observations, and nuclear norm minimization (NNM) has been widely used as a popular heuristic of rank minimization for low-rank matrix recovery problems. On the other hand, it has been shown that NNM can be viewed as a special case of atomic norm minimization (ANM), which has achieved great success in solving line spectrum estimation problems. However, as far as we know, the general ANM (not NNM) considered in many existing works can only handle frequency estimation in undamped sinusoids. In this work, we aim to fill this gap and deal with damped spectrally sparse signal recovery problems. In particular, inspired by the dual analysis used in ANM, we offer a novel optimization-based perspective on the classical MUSIC algorithm and propose an algorithm for spectral estimation that involves searching for the peaks of the dual polynomial corresponding to a certain NNM problem, and we show that this algorithm is in fact equivalent to MUSIC itself. Building on this connection, we also extend the classical MUSIC algorithm to the missing data case. We provide exact recovery guarantees for our proposed algorithms and quantify how the sample complexity depends on the true spectral parameters. In particular, we provide a parameter-specific recovery bound for low-rank matrix recovery of jointly sparse signals rather than use certain incoherence properties as in existing literature. Simulation results also indicate that the proposed algorithms significantly outperform some relevant existing methods (e.g., ANM) in frequency estimation of damped exponentials.
△ Less
Submitted 19 August, 2020; v1 submitted 9 June, 2018;
originally announced June 2018.
-
Sparse Blind Deconvolution for Distributed Radar Autofocus Imaging
Authors:
Hassan Mansour,
Dehong Liu,
Ulugbek S. Kamilov,
Petros T. Boufounos
Abstract:
A common problem that arises in radar imaging systems, especially those mounted on mobile platforms, is antenna position ambiguity. Approaches to resolve this ambiguity and correct position errors are generally known as radar autofocus. Common techniques that attempt to resolve the antenna ambiguity generally assume an unknown gain and phase error afflicting the radar measurements. However, ensuri…
▽ More
A common problem that arises in radar imaging systems, especially those mounted on mobile platforms, is antenna position ambiguity. Approaches to resolve this ambiguity and correct position errors are generally known as radar autofocus. Common techniques that attempt to resolve the antenna ambiguity generally assume an unknown gain and phase error afflicting the radar measurements. However, ensuring identifiability and tractability of the unknown error imposes strict restrictions on the allowable antenna perturbations. Furthermore, these techniques are often not applicable in near-field imaging, where map** the position ambiguity to phase errors breaks down.
In this paper, we propose an alternate formulation where the position error of each antenna is mapped to a spatial shift operator in the image-domain. Thus, the radar autofocus problem becomes a multichannel blind deconvolution problem, in which the radar measurements correspond to observations of a static radar image that is convolved with the spatial shift kernel associated with each antenna. To solve the reformulated problem, we also develop a block coordinate descent framework that leverages the sparsity and piece-wise smoothness of the radar scene, as well as the one-sparse property of the two dimensional shift kernels. We evaluate the performance of our approach using both simulated and experimental radar measurements, and demonstrate its superior performance compared to state-of-the-art methods.
△ Less
Submitted 8 May, 2018;
originally announced May 2018.
-
Online Detection of Action Start in Untrimmed, Streaming Videos
Authors:
Zheng Shou,
Junting Pan,
Jonathan Chan,
Kazuyuki Miyazawa,
Hassan Mansour,
Anthony Vetro,
Xavier Giro-i-Nieto,
Shih-Fu Chang
Abstract:
We aim to tackle a novel task in action detection - Online Detection of Action Start (ODAS) in untrimmed, streaming videos. The goal of ODAS is to detect the start of an action instance, with high categorization accuracy and low detection latency. ODAS is important in many applications such as early alert generation to allow timely security or emergency response. We propose three novel methods to…
▽ More
We aim to tackle a novel task in action detection - Online Detection of Action Start (ODAS) in untrimmed, streaming videos. The goal of ODAS is to detect the start of an action instance, with high categorization accuracy and low detection latency. ODAS is important in many applications such as early alert generation to allow timely security or emergency response. We propose three novel methods to specifically address the challenges in training ODAS models: (1) hard negative samples generation based on Generative Adversarial Network (GAN) to distinguish ambiguous background, (2) explicitly modeling the temporal consistency between data around action start and data succeeding action start, and (3) adaptive sampling strategy to handle the scarcity of training data. We conduct extensive experiments using THUMOS'14 and ActivityNet. We show that our proposed methods lead to significant performance gains and improve the state-of-the-art methods. An ablation study confirms the effectiveness of each proposed method.
△ Less
Submitted 23 July, 2018; v1 submitted 19 February, 2018;
originally announced February 2018.
-
Accelerated Image Reconstruction for Nonlinear Diffractive Imaging
Authors:
Yanting Ma,
Hassan Mansour,
Dehong Liu,
Petros T. Boufounos,
Ulugbek S. Kamilov
Abstract:
The problem of reconstructing an object from the measurements of the light it scatters is common in numerous imaging applications. While the most popular formulations of the problem are based on linearizing the object-light relationship, there is an increased interest in considering nonlinear formulations that can account for multiple light scattering. In this paper, we propose an image reconstruc…
▽ More
The problem of reconstructing an object from the measurements of the light it scatters is common in numerous imaging applications. While the most popular formulations of the problem are based on linearizing the object-light relationship, there is an increased interest in considering nonlinear formulations that can account for multiple light scattering. In this paper, we propose an image reconstruction method, called CISOR, for nonlinear diffractive imaging, based on a nonconvex optimization formulation with total variation (TV) regularization. The nonconvex solver used in CISOR is our new variant of fast iterative shrinkage/thresholding algorithm (FISTA). We provide fast and memory-efficient implementation of the new FISTA variant and prove that it reliably converges for our nonconvex optimization problem. In addition, we systematically compare our method with other state-of-the-art methods on simulated as well as experimentally measured data in both 2D and 3D settings.
△ Less
Submitted 14 December, 2017; v1 submitted 4 August, 2017;
originally announced August 2017.
-
SEAGLE: Sparsity-Driven Image Reconstruction under Multiple Scattering
Authors:
Hsiou-Yuan Liu,
Dehong Liu,
Hassan Mansour,
Petros T. Boufounos,
Laura Waller,
Ulugbek S. Kamilov
Abstract:
Multiple scattering of an electromagnetic wave as it passes through an object is a fundamental problem that limits the performance of current imaging systems. In this paper, we describe a new technique-called Series Expansion with Accelerated Gradient Descent on Lippmann-Schwinger Equation (SEAGLE)-for robust imaging under multiple scattering based on a combination of a new nonlinear forward model…
▽ More
Multiple scattering of an electromagnetic wave as it passes through an object is a fundamental problem that limits the performance of current imaging systems. In this paper, we describe a new technique-called Series Expansion with Accelerated Gradient Descent on Lippmann-Schwinger Equation (SEAGLE)-for robust imaging under multiple scattering based on a combination of a new nonlinear forward model and a total variation (TV) regularizer. The proposed forward model can account for multiple scattering, which makes it advantageous in applications where linear models are inaccurate. Specifically, it corresponds to a series expansion of the scattered wave with an accelerated-gradient method. This expansion guarantees the convergence even for strongly scattering objects. One of our key insights is that it is possible to obtain an explicit formula for computing the gradient of our nonlinear forward model with respect to the unknown object, thus enabling fast image reconstruction with the state-of-the-art fast iterative shrinkage/thresholding algorithm (FISTA). The proposed method is validated on both simulated and experimentally measured data.
△ Less
Submitted 5 May, 2017;
originally announced May 2017.
-
Guided Signal Reconstruction Theory
Authors:
Andrew Knyazev,
Akshay Gadde,
Hassan Mansour,
Dong Tian
Abstract:
An axiomatic approach to signal reconstruction is formulated, involving a sample consistent set and a guiding set, describing desired reconstructions. New frame-less reconstruction methods are proposed, based on a novel concept of a reconstruction set, defined as a shortest pathway between the sample consistent set and the guiding set. Existence and uniqueness of the reconstruction set are investi…
▽ More
An axiomatic approach to signal reconstruction is formulated, involving a sample consistent set and a guiding set, describing desired reconstructions. New frame-less reconstruction methods are proposed, based on a novel concept of a reconstruction set, defined as a shortest pathway between the sample consistent set and the guiding set. Existence and uniqueness of the reconstruction set are investigated in a Hilbert space, where the guiding set is a closed subspace and the sample consistent set is a closed plane, formed by a sampling subspace. Connections to earlier known consistent, generalized, and regularized reconstructions are clarified. New stability and reconstruction error bounds are derived, using the largest nontrivial angle between the sampling and guiding subspaces. Conjugate gradient iterative reconstruction algorithms are proposed and illustrated numerically for image magnification.
△ Less
Submitted 2 February, 2017;
originally announced February 2017.
-
Compressive Imaging with Iterative Forward Models
Authors:
Hsiou-Yuan Liu,
Ulugbek S. Kamilov,
Dehong Liu,
Hassan Mansour,
Petros T. Boufounos
Abstract:
We propose a new compressive imaging method for reconstructing 2D or 3D objects from their scattered wave-field measurements. Our method relies on a novel, nonlinear measurement model that can account for the multiple scattering phenomenon, which makes the method preferable in applications where linear measurement models are inaccurate. We construct the measurement model by expanding the scattered…
▽ More
We propose a new compressive imaging method for reconstructing 2D or 3D objects from their scattered wave-field measurements. Our method relies on a novel, nonlinear measurement model that can account for the multiple scattering phenomenon, which makes the method preferable in applications where linear measurement models are inaccurate. We construct the measurement model by expanding the scattered wave-field with an accelerated-gradient method, which is guaranteed to converge and is suitable for large-scale problems. We provide explicit formulas for computing the gradient of our measurement model with respect to the unknown image, which enables image formation with a sparsity- driven numerical optimization algorithm. We validate the method both analytically and with numerical simulations.
△ Less
Submitted 5 October, 2016;
originally announced October 2016.
-
Proceedings of the third "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'16)
Authors:
V. Abrol,
O. Absil,
P. -A. Absil,
S. Anthoine,
P. Antoine,
T. Arildsen,
N. Bertin,
F. Bleichrodt,
J. Bobin,
A. Bol,
A. Bonnefoy,
F. Caltagirone,
V. Cambareri,
C. Chenot,
V. Crnojević,
M. Daňková,
K. Degraux,
J. Eisert,
J. M. Fadili,
M. Gabrié,
N. Gac,
D. Giacobello,
A. Gonzalez,
C. A. Gomez Gonzalez,
A. González
, et al. (36 additional authors not shown)
Abstract:
The third edition of the "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) took place in Aalborg, the 4th largest city in Denmark situated beautifully in the northern part of the country, from the 24th to 26th of August 2016. The workshop venue was at the Aalborg University campus. One implicit objective of this biennial workshop is to foster collab…
▽ More
The third edition of the "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) took place in Aalborg, the 4th largest city in Denmark situated beautifully in the northern part of the country, from the 24th to 26th of August 2016. The workshop venue was at the Aalborg University campus. One implicit objective of this biennial workshop is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For this third edition, iTWIST'16 gathered about 50 international participants and features 8 invited talks, 12 oral presentations, and 12 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing (e.g., optics, computer vision, genomics, biomedical, digital communication, channel estimation, astronomy); Application of sparse models in non-convex/non-linear inverse problems (e.g., phase retrieval, blind deconvolution, self calibration); Approximate probabilistic inference for sparse problems; Sparse machine learning and inference; "Blind" inverse problems and dictionary learning; Optimization for sparse modelling; Information theory, geometry and randomness; Sparsity? What's next? (Discrete-valued signals; Union of low-dimensional spaces, Cosparsity, mixed/group norm, model-based, low-complexity models, ...); Matrix/manifold sensing/processing (graph, low-rank approximation, ...); Complexity/accuracy tradeoffs in numerical methods/optimization; Electronic/optical compressive sensors (hardware).
△ Less
Submitted 14 September, 2016;
originally announced September 2016.
-
A Recursive Born Approach to Nonlinear Inverse Scattering
Authors:
Ulugbek S. Kamilov,
Dehong Liu,
Hassan Mansour,
Petros T. Boufounos
Abstract:
The Iterative Born Approximation (IBA) is a well-known method for describing waves scattered by semi-transparent objects. In this paper, we present a novel nonlinear inverse scattering method that combines IBA with an edge-preserving total variation (TV) regularizer. The proposed method is obtained by relating iterations of IBA to layers of a feedforward neural network and develo** a correspondi…
▽ More
The Iterative Born Approximation (IBA) is a well-known method for describing waves scattered by semi-transparent objects. In this paper, we present a novel nonlinear inverse scattering method that combines IBA with an edge-preserving total variation (TV) regularizer. The proposed method is obtained by relating iterations of IBA to layers of a feedforward neural network and develo** a corresponding error backpropagation algorithm for efficiently estimating the permittivity of the object. Simulations illustrate that, by accounting for multiple scattering, the method successfully recovers the permittivity distribution where the traditional linear inverse scattering fails.
△ Less
Submitted 11 March, 2016;
originally announced March 2016.
-
Representation and Coding of Signal Geometry
Authors:
Petros T Boufounos,
Shantanu Rane,
Hassan Mansour
Abstract:
Approaches to signal representation and coding theory have traditionally focused on how to best represent signals using parsimonious representations that incur the lowest possible distortion. Classical examples include linear and non-linear approximations, sparse representations, and rate-distortion theory. Very often, however, the goal of processing is to extract specific information from the sig…
▽ More
Approaches to signal representation and coding theory have traditionally focused on how to best represent signals using parsimonious representations that incur the lowest possible distortion. Classical examples include linear and non-linear approximations, sparse representations, and rate-distortion theory. Very often, however, the goal of processing is to extract specific information from the signal, and the distortion should be measured on the extracted information. The corresponding representation should, therefore, represent that information as parsimoniously as possible, without necessarily accurately representing the signal itself.
In this paper, we examine the problem of encoding signals such that sufficient information is preserved about their pairwise distances and their inner products. For that goal, we consider randomized embeddings as an encoding mechanism and provide a framework to analyze their performance. We also demonstrate that it is possible to design the embedding such that it represents different ranges of distances with different precision. These embeddings also allow the computation of kernel inner products with control on their inner product-preserving properties. Our results provide a broad framework to design and analyze embeddins, and generalize existing results in this area, such as random Fourier kernels and universal embeddings.
△ Less
Submitted 23 December, 2015;
originally announced December 2015.
-
Learning optimal nonlinearities for iterative thresholding algorithms
Authors:
Ulugbek S. Kamilov,
Hassan Mansour
Abstract:
Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding functions for ISTA. The proposed scheme is obtained by relating iterations of ISTA to layers of a simple deep neural network (DNN) and develo** a corresponding error backpropagation…
▽ More
Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding functions for ISTA. The proposed scheme is obtained by relating iterations of ISTA to layers of a simple deep neural network (DNN) and develo** a corresponding error backpropagation algorithm that allows to fine-tune the thresholding functions. Simulations on sparse statistical signals illustrate potential gains in estimation quality due to the proposed data adaptive ISTA.
△ Less
Submitted 15 December, 2015;
originally announced December 2015.
-
Guided Signal Reconstruction with Application to Image Magnification
Authors:
Akshay Gadde,
Andrew Knyazev,
Dong Tian,
Hassan Mansour
Abstract:
We study the problem of reconstructing a signal from its projection on a subspace. The proposed signal reconstruction algorithms utilize a guiding subspace that represents desired properties of reconstructed signals. We show that optimal reconstructed signals belong to a convex bounded set, called the "reconstruction" set. We also develop iterative algorithms, based on conjugate gradient methods,…
▽ More
We study the problem of reconstructing a signal from its projection on a subspace. The proposed signal reconstruction algorithms utilize a guiding subspace that represents desired properties of reconstructed signals. We show that optimal reconstructed signals belong to a convex bounded set, called the "reconstruction" set. We also develop iterative algorithms, based on conjugate gradient methods, to approximate optimal reconstructions with low memory and computational costs. The effectiveness of the proposed approach is demonstrated for image magnification, where the reconstructed image quality is shown to exceed that of consistent and generalized reconstruction schemes.
△ Less
Submitted 8 September, 2015;
originally announced September 2015.
-
Chebyshev and Conjugate Gradient Filters for Graph Image Denoising
Authors:
Dong Tian,
Hassan Mansour,
Andrew Knyazev,
Anthony Vetro
Abstract:
In 3D image/video acquisition, different views are often captured with varying noise levels across the views. In this paper, we propose a graph-based image enhancement technique that uses a higher quality view to enhance a degraded view. A depth map is utilized as auxiliary information to match the perspectives of the two views. Our method performs graph-based filtering of the noisy image by direc…
▽ More
In 3D image/video acquisition, different views are often captured with varying noise levels across the views. In this paper, we propose a graph-based image enhancement technique that uses a higher quality view to enhance a degraded view. A depth map is utilized as auxiliary information to match the perspectives of the two views. Our method performs graph-based filtering of the noisy image by directly computing a projection of the image to be filtered onto a lower dimensional Krylov subspace of the graph Laplacian. We discuss two graph spectral denoising methods: first using Chebyshev polynomials, and second using iterations of the conjugate gradient algorithm. Our framework generalizes previously known polynomial graph filters, and we demonstrate through numerical simulations that our proposed technique produces subjectively cleaner images with about 1-3 dB improvement in PSNR over existing polynomial graph filters.
△ Less
Submitted 4 September, 2015;
originally announced September 2015.
-
Facial Expressions recognition Based on Principal Component Analysis (PCA)
Authors:
Abdelmajid Hassan Mansour,
Gafar Zen Alabdeen Salh,
Ali Shaif Alhalemi
Abstract:
The facial expression recognition is an ocular task that can be performed without human discomfort, is really a speedily growing on the computer research field. There are many applications and programs uses facial expression to evaluate human character, judgment, feelings, and viewpoint. The process of recognizing facial expression is a hard task due to the several circumstances such as facial occ…
▽ More
The facial expression recognition is an ocular task that can be performed without human discomfort, is really a speedily growing on the computer research field. There are many applications and programs uses facial expression to evaluate human character, judgment, feelings, and viewpoint. The process of recognizing facial expression is a hard task due to the several circumstances such as facial occlusions, face shape, illumination, face colors, and etc. This paper present a PCA methodology to distinguish expressions of faces under different circumstances and identifying it. Relies on Eigen faces technique using standard Data base images. So as to overcome the problem of difficulty to computers to identify the features and expressions of persons.
△ Less
Submitted 23 December, 2014;
originally announced June 2015.
-
Recovery Analysis for Weighted $\ell_1$-Minimization Using a Null Space Property
Authors:
Hassan Mansour,
Rayan Saab
Abstract:
We study the recovery of sparse signals from underdetermined linear measurements when a potentially erroneous support estimate is available. Our results are twofold. First, we derive necessary and sufficient conditions for signal recovery from compressively sampled measurements using weighted $\ell_1$-norm minimization. These conditions, which depend on the choice of weights as well as the size an…
▽ More
We study the recovery of sparse signals from underdetermined linear measurements when a potentially erroneous support estimate is available. Our results are twofold. First, we derive necessary and sufficient conditions for signal recovery from compressively sampled measurements using weighted $\ell_1$-norm minimization. These conditions, which depend on the choice of weights as well as the size and accuracy of the support estimate, are on the null space of the measurement matrix. They can guarantee recovery even when standard $\ell_1$ minimization fails. Second, we derive bounds on the number of Gaussian measurements for these conditions to be satisfied, i.e., for weighted $\ell_1$ minimization to successfully recover all sparse signals whose support has been estimated sufficiently accurately. Our bounds show that weighted $\ell_1$ minimization requires significantly fewer measurements than standard $\ell_1$ minimization when the support estimate is relatively accurate.
△ Less
Submitted 4 December, 2014;
originally announced December 2014.
-
Non-Convex Compressed Sensing Using Partial Support Information
Authors:
Navid Ghadermarzy,
Hassan Mansour,
Ozgur Yilmaz
Abstract:
In this paper we address the recovery conditions of weighted $\ell_p$ minimization for signal reconstruction from compressed sensing measurements when partial support information is available. We show that weighted $\ell_p$ minimization with $0<p<1$ is stable and robust under weaker sufficient conditions compared to weighted $\ell_1$ minimization. Moreover, the sufficient recovery conditions of we…
▽ More
In this paper we address the recovery conditions of weighted $\ell_p$ minimization for signal reconstruction from compressed sensing measurements when partial support information is available. We show that weighted $\ell_p$ minimization with $0<p<1$ is stable and robust under weaker sufficient conditions compared to weighted $\ell_1$ minimization. Moreover, the sufficient recovery conditions of weighted $\ell_p$ are weaker than those of regular $\ell_p$ minimization if at least $50%$ of the support estimate is accurate. We also review some algorithms which exist to solve the non-convex $\ell_p$ problem and illustrate our results with numerical experiments.
△ Less
Submitted 15 November, 2013;
originally announced November 2013.
-
A fast randomized Kaczmarz algorithm for sparse solutions of consistent linear systems
Authors:
Hassan Mansour,
Ozgur Yilmaz
Abstract:
The Kaczmarz algorithm is a popular solver for overdetermined linear systems due to its simplicity and speed. In this paper, we propose a modification that speeds up the convergence of the randomized Kaczmarz algorithm for systems of linear equations with sparse solutions. The speedup is achieved by projecting every iterate onto a weighted row of the linear system while maintaining the random row…
▽ More
The Kaczmarz algorithm is a popular solver for overdetermined linear systems due to its simplicity and speed. In this paper, we propose a modification that speeds up the convergence of the randomized Kaczmarz algorithm for systems of linear equations with sparse solutions. The speedup is achieved by projecting every iterate onto a weighted row of the linear system while maintaining the random row selection criteria of Strohmer and Vershynin. The weights are chosen to attenuate the contribution of row elements that lie outside of the estimated support of the sparse solution. While the Kaczmarz algorithm and its variants can only find solutions to overdetermined linear systems, our algorithm surprisingly succeeds in finding sparse solutions to underdetermined linear systems as well. We present empirical studies which demonstrate the acceleration in convergence to the sparse solution using this modified approach in the overdetermined case. We also demonstrate the sparse recovery capabilities of our approach in the underdetermined case and compare the performance with that of $\ell_1$ minimization.
△ Less
Submitted 16 May, 2013;
originally announced May 2013.
-
Fast methods for denoising matrix completion formulations, with applications to robust seismic data interpolation
Authors:
Aleksandr Y. Aravkin,
Rajiv Kumar,
Hassan Mansour,
Ben Recht,
Felix J. Herrmann
Abstract:
Recent SVD-free matrix factorization formulations have enabled rank minimization for systems with millions of rows and columns, paving the way for matrix completion in extremely large-scale applications, such as seismic data interpolation.
In this paper, we consider matrix completion formulations designed to hit a target data-fitting error level provided by the user, and propose an algorithm cal…
▽ More
Recent SVD-free matrix factorization formulations have enabled rank minimization for systems with millions of rows and columns, paving the way for matrix completion in extremely large-scale applications, such as seismic data interpolation.
In this paper, we consider matrix completion formulations designed to hit a target data-fitting error level provided by the user, and propose an algorithm called LR-BPDN that is able to exploit factorized formulations to solve the corresponding optimization problem. Since practitioners typically have strong prior knowledge about target error level, this innovation makes it easy to apply the algorithm in practice, leaving only the factor rank to be determined.
Within the established framework, we propose two extensions that are highly relevant to solving practical challenges of data interpolation. First, we propose a weighted extension that allows known subspace information to improve the results of matrix completion formulations. We show how this weighting can be used in the context of frequency continuation, an essential aspect to seismic data interpolation. Second, we propose matrix completion formulations that are robust to large measurement errors in the available data.
We illustrate the advantages of LR-BPDN on the collaborative filtering problem using the MovieLens 1M, 10M, and Netflix 100M datasets. Then, we use the new method, along with its robust and subspace re-weighted extensions, to obtain high-quality reconstructions for large scale seismic interpolation problems with real data, even in the presence of data contamination.
△ Less
Submitted 5 March, 2014; v1 submitted 20 February, 2013;
originally announced February 2013.
-
Beyond $\ell_1$-norm minimization for sparse signal recovery
Authors:
Hassan Mansour
Abstract:
Sparse signal recovery has been dominated by the basis pursuit denoise (BPDN) problem formulation for over a decade. In this paper, we propose an algorithm that outperforms BPDN in finding sparse solutions to underdetermined linear systems of equations at no additional computational cost. Our algorithm, called WSPGL1, is a modification of the spectral projected gradient for $\ell_1$ minimization (…
▽ More
Sparse signal recovery has been dominated by the basis pursuit denoise (BPDN) problem formulation for over a decade. In this paper, we propose an algorithm that outperforms BPDN in finding sparse solutions to underdetermined linear systems of equations at no additional computational cost. Our algorithm, called WSPGL1, is a modification of the spectral projected gradient for $\ell_1$ minimization (SPGL1) algorithm in which the sequence of LASSO subproblems are replaced by a sequence of weighted LASSO subproblems with constant weights applied to a support estimate. The support estimate is derived from the data and is updated at every iteration. The algorithm also modifies the Pareto curve at every iteration to reflect the new weighted $\ell_1$ minimization problem that is being solved. We demonstrate through extensive simulations that the sparse recovery performance of our algorithm is superior to that of $\ell_1$ minimization and approaches the recovery performance of iterative re-weighted $\ell_1$ (IRWL1) minimization of Cand{è}s, Wakin, and Boyd, although it does not match it in general. Moreover, our algorithm has the computational cost of a single BPDN problem.
△ Less
Submitted 30 May, 2012;
originally announced May 2012.
-
Support driven reweighted $\ell_1$ minimization
Authors:
Hassan Mansour,
Ozgur Yilmaz
Abstract:
In this paper, we propose a support driven reweighted $\ell_1$ minimization algorithm (SDRL1) that solves a sequence of weighted $\ell_1$ problems and relies on the support estimate accuracy. Our SDRL1 algorithm is related to the IRL1 algorithm proposed by Cand{è}s, Wakin, and Boyd. We demonstrate that it is sufficient to find support estimates with \emph{good} accuracy and apply constant weights…
▽ More
In this paper, we propose a support driven reweighted $\ell_1$ minimization algorithm (SDRL1) that solves a sequence of weighted $\ell_1$ problems and relies on the support estimate accuracy. Our SDRL1 algorithm is related to the IRL1 algorithm proposed by Cand{è}s, Wakin, and Boyd. We demonstrate that it is sufficient to find support estimates with \emph{good} accuracy and apply constant weights instead of using the inverse coefficient magnitudes to achieve gains similar to those of IRL1. We then prove that given a support estimate with sufficient accuracy, if the signal decays according to a specific rate, the solution to the weighted $\ell_1$ minimization problem results in a support estimate with higher accuracy than the initial estimate. We also show that under certain conditions, it is possible to achieve higher estimate accuracy when the intersection of support estimates is considered. We demonstrate the performance of SDRL1 through numerical simulations and compare it with that of IRL1 and standard $\ell_1$ minimization.
△ Less
Submitted 30 May, 2012;
originally announced May 2012.
-
Weighted-{$\ell_1$} minimization with multiple weighting sets
Authors:
Hassan Mansour,
Ozgur Yilmaz
Abstract:
In this paper, we study the support recovery conditions of weighted $\ell_1$ minimization for signal reconstruction from compressed sensing measurements when multiple support estimate sets with different accuracy are available. We identify a class of signals for which the recovered vector from $\ell_1$ minimization provides an accurate support estimate. We then derive stability and robustness guar…
▽ More
In this paper, we study the support recovery conditions of weighted $\ell_1$ minimization for signal reconstruction from compressed sensing measurements when multiple support estimate sets with different accuracy are available. We identify a class of signals for which the recovered vector from $\ell_1$ minimization provides an accurate support estimate. We then derive stability and robustness guarantees for the weighted $\ell_1$ minimization problem with more than one support estimate. We show that applying a smaller weight to support estimate that enjoy higher accuracy improves the recovery conditions compared with the case of a single support estimate and the case with standard, i.e., non-weighted, $\ell_1$ minimization. Our theoretical results are supported by numerical simulations on synthetic signals and real audio signals.
△ Less
Submitted 30 May, 2012;
originally announced May 2012.
-
Recovering Compressively Sampled Signals Using Partial Support Information
Authors:
Michael P. Friedlander,
Hassan Mansour,
Rayan Saab,
Ozgur Yilmaz
Abstract:
In this paper we study recovery conditions of weighted $\ell_1$ minimization for signal reconstruction from compressed sensing measurements when partial support information is available. We show that if at least 50% of the (partial) support information is accurate, then weighted $\ell_1$ minimization is stable and robust under weaker conditions than the analogous conditions for standard $\ell_1$ m…
▽ More
In this paper we study recovery conditions of weighted $\ell_1$ minimization for signal reconstruction from compressed sensing measurements when partial support information is available. We show that if at least 50% of the (partial) support information is accurate, then weighted $\ell_1$ minimization is stable and robust under weaker conditions than the analogous conditions for standard $\ell_1$ minimization. Moreover, weighted $\ell_1$ minimization provides better bounds on the reconstruction error in terms of the measurement noise and the compressibility of the signal to be recovered. We illustrate our results with extensive numerical experiments on synthetic data and real audio and video signals.
△ Less
Submitted 22 July, 2011; v1 submitted 22 October, 2010;
originally announced October 2010.
