-
Improving embedding of graphs with missing data by soft manifolds
Authors:
Andrea Marinoni,
Pietro Lio',
Alessandro Barp,
Christian Jutten,
Mark Girolami
Abstract:
Embedding graphs in continous spaces is a key factor in designing and develo** algorithms for automatic information extraction to be applied in diverse tasks (e.g., learning, inferring, predicting). The reliability of graph embeddings directly depends on how much the geometry of the continuous space matches the graph structure. Manifolds are mathematical structure that can enable to incorporate…
▽ More
Embedding graphs in continous spaces is a key factor in designing and develo** algorithms for automatic information extraction to be applied in diverse tasks (e.g., learning, inferring, predicting). The reliability of graph embeddings directly depends on how much the geometry of the continuous space matches the graph structure. Manifolds are mathematical structure that can enable to incorporate in their topological spaces the graph characteristics, and in particular nodes distances. State-of-the-art of manifold-based graph embedding algorithms take advantage of the assumption that the projection on a tangential space of each point in the manifold (corresponding to a node in the graph) would locally resemble a Euclidean space. Although this condition helps in achieving efficient analytical solutions to the embedding problem, it does not represent an adequate set-up to work with modern real life graphs, that are characterized by weighted connections across nodes often computed over sparse datasets with missing records. In this work, we introduce a new class of manifold, named soft manifold, that can solve this situation. In particular, soft manifolds are mathematical structures with spherical symmetry where the tangent spaces to each point are hypocycloids whose shape is defined according to the velocity of information propagation across the data points. Using soft manifolds for graph embedding, we can provide continuous spaces to pursue any task in data analysis over complex datasets. Experimental results on reconstruction tasks on synthetic and real datasets show how the proposed approach enable more accurate and reliable characterization of graphs in continuous spaces with respect to the state-of-the-art.
△ Less
Submitted 29 November, 2023;
originally announced November 2023.
-
Electrode Selection for Noninvasive Fetal Electrocardiogram Extraction using Mutual Information Criteria
Authors:
Reza Sameni,
Frédéric Vrins,
Fabienne Parmentier,
Christophe Hérail,
Vincent Vigneron,
Michel Verleysen,
Christian Jutten,
Mohammad B. Shamsollahi
Abstract:
Blind source separation (BSS) techniques have revealed to be promising approaches for, among other, biomedical signal processing applications. Specifically, for the noninvasive extraction of fetal cardiac signals from maternal abdominal recordings, where conventional filtering schemes have failed to extract the complete fetal ECG components. From previous studies, it is now believed that a careful…
▽ More
Blind source separation (BSS) techniques have revealed to be promising approaches for, among other, biomedical signal processing applications. Specifically, for the noninvasive extraction of fetal cardiac signals from maternal abdominal recordings, where conventional filtering schemes have failed to extract the complete fetal ECG components. From previous studies, it is now believed that a carefully selected array of electrodes well-placed over the abdomen of a pregnant woman contains the required `information' for BSS, to extract the complete fetal components. Based on this idea, in previous works array recording systems and sensor selection strategies based on the Mutual Information (MI) criterion have been developed. In this paper the previous works have been extended, by considering the 3-dimensional aspects of the cardiac electrical activity. The proposed method has been tested on simulated and real maternal abdominal recordings. The results show that the new sensor selection strategy together with the MI criterion, can be effectively used to select the channels containing the most `information' concerning the fetal ECG components from an array of 72 recordings. The method is hence believed to be useful for the selection of the most informative channels in online applications, considering the different fetal positions and movements.
△ Less
Submitted 31 January, 2023;
originally announced February 2023.
-
A graph representation based on fluid diffusion model for data analysis: theoretical aspects and enhanced community detection
Authors:
Andrea Marinoni,
Christian Jutten,
Mark Girolami
Abstract:
Representing data by means of graph structures identifies one of the most valid approach to extract information in several data analysis applications. This is especially true when multimodal datasets are investigated, as records collected by means of diverse sensing strategies are taken into account and explored. Nevertheless, classic graph signal processing is based on a model for information pro…
▽ More
Representing data by means of graph structures identifies one of the most valid approach to extract information in several data analysis applications. This is especially true when multimodal datasets are investigated, as records collected by means of diverse sensing strategies are taken into account and explored. Nevertheless, classic graph signal processing is based on a model for information propagation that is configured according to heat diffusion mechanism. This system provides several constraints and assumptions on the data properties that might be not valid for multimodal data analysis, especially when large scale datasets collected from heterogeneous sources are considered, so that the accuracy and robustness of the outcomes might be severely jeopardized. In this paper, we introduce a novel model for graph definition based on fluid diffusion. The proposed approach improves the ability of graph-based data analysis to take into account several issues of modern data analysis in operational scenarios, so to provide a platform for precise, versatile, and efficient understanding of the phenomena underlying the records under exam, and to fully exploit the potential provided by the diversity of the records in obtaining a thorough characterization of the data and their significance. In this work, we focus our attention to using this fluid diffusion model to drive a community detection scheme, i.e., to divide multimodal datasets into many groups according to similarity among nodes in an unsupervised fashion. Experimental results achieved by testing real multimodal datasets in diverse application scenarios show that our method is able to strongly outperform state-of-the-art schemes for community detection in multimodal data analysis.
△ Less
Submitted 17 October, 2022; v1 submitted 7 December, 2021;
originally announced December 2021.
-
Temporally Nonstationary Component Analysis; Application to Noninvasive Fetal Electrocardiogram Extraction
Authors:
Fahimeh Jamshidian-Tehrani,
Reza Sameni,
Christian Jutten
Abstract:
Objective: Mixtures of temporally nonstationary signals are very common in biomedical applications. The nonstationarity of the source signals can be used as a discriminative property for signal separation. Herein, a semi-blind source separation algorithm is proposed for the extraction of temporally nonstationary components from linear multichannel mixtures of signals and noises. Methods: A hypothe…
▽ More
Objective: Mixtures of temporally nonstationary signals are very common in biomedical applications. The nonstationarity of the source signals can be used as a discriminative property for signal separation. Herein, a semi-blind source separation algorithm is proposed for the extraction of temporally nonstationary components from linear multichannel mixtures of signals and noises. Methods: A hypothesis test is proposed for the detection and fusion of temporally nonstationary events, by using ad hoc indexes for monitoring the first and second order statistics of the innovation process. As proof of concept, the general framework is customized and tested over noninvasive fetal cardiac recordings acquired from the maternal abdomen, over publicly available datasets, using two types of nonstationarity detectors: 1) a local power variations detector, and 2) a model-deviations detector using the innovation process properties of an extended Kalman filter. Results: The performance of the proposed method is assessed in presence of white and colored noise, in different signal-to-noise ratios. Conclusion and Significance: The proposed scheme is general and it can be used for the extraction of nonstationary events and sample deviations from a presumed model in multivariate data, which is a recurrent problem in many machine learning applications.
△ Less
Submitted 20 August, 2021;
originally announced August 2021.
-
A Hypothesis Testing Approach to Nonstationary Source Separation
Authors:
Reza Sameni,
Christian Jutten
Abstract:
The extraction of nonstationary signals from blind and semi-blind multivariate observations is a recurrent problem. Numerous algorithms have been developed for this problem, which are based on the exact or approximate joint diagonalization of second or higher order cumulant matrices/tensors of multichannel data. While a great body of research has been dedicated to joint diagonalization algorithms,…
▽ More
The extraction of nonstationary signals from blind and semi-blind multivariate observations is a recurrent problem. Numerous algorithms have been developed for this problem, which are based on the exact or approximate joint diagonalization of second or higher order cumulant matrices/tensors of multichannel data. While a great body of research has been dedicated to joint diagonalization algorithms, the selection of the diagonalized matrix/tensor set remains highly problem-specific. Herein, various methods for nonstationarity identification are reviewed and a new general framework based on hypothesis testing is proposed, which results in a classification/clustering perspective to semi-blind source separation of nonstationary components. The proposed method is applied to noninvasive fetal ECG extraction, as case study.
△ Less
Submitted 28 June, 2021; v1 submitted 14 May, 2021;
originally announced May 2021.
-
Enhancing ensemble learning and transfer learning in multimodal data analysis by adaptive dimensionality reduction
Authors:
Andrea Marinoni,
Saloua Chlaily,
Eduard Khachatrian,
Torbjørn Eltoft,
Sivasakthy Selvakumaran,
Mark Girolami,
Christian Jutten
Abstract:
Modern data analytics take advantage of ensemble learning and transfer learning approaches to tackle some of the most relevant issues in data analysis, such as lack of labeled data to use to train the analysis models, sparsity of the information, and unbalanced distributions of the records. Nonetheless, when applied to multimodal datasets (i.e., datasets acquired by means of multiple sensing techn…
▽ More
Modern data analytics take advantage of ensemble learning and transfer learning approaches to tackle some of the most relevant issues in data analysis, such as lack of labeled data to use to train the analysis models, sparsity of the information, and unbalanced distributions of the records. Nonetheless, when applied to multimodal datasets (i.e., datasets acquired by means of multiple sensing techniques or strategies), the state-of-theart methods for ensemble learning and transfer learning might show some limitations. In fact, in multimodal data analysis, not all observations would show the same level of reliability or information quality, nor an homogeneous distribution of errors and uncertainties. This condition might undermine the classic assumptions ensemble learning and transfer learning methods rely on. In this work, we propose an adaptive approach for dimensionality reduction to overcome this issue. By means of a graph theory-based approach, the most relevant features across variable size subsets of the considered datasets are identified. This information is then used to set-up ensemble learning and transfer learning architectures. We test our approach on multimodal datasets acquired in diverse research fields (remote sensing, brain-computer interfaces, photovoltaic energy). Experimental results show the validity and the robustness of our approach, able to outperform state-of-the-art techniques.
△ Less
Submitted 8 May, 2021;
originally announced May 2021.
-
Spectral Variability in Hyperspectral Data Unmixing: A Comprehensive Review
Authors:
Ricardo Augusto Borsoi,
Tales Imbiriba,
José Carlos Moreira Bermudez,
Cédric Richard,
Jocelyn Chanussot,
Lucas Drumetz,
Jean-Yves Tourneret,
Alina Zare,
Christian Jutten
Abstract:
The spectral signatures of the materials contained in hyperspectral images, also called endmembers (EM), can be significantly affected by variations in atmospheric, illumination or environmental conditions typically occurring within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, what propagates significant mismodeling errors throughout t…
▽ More
The spectral signatures of the materials contained in hyperspectral images, also called endmembers (EM), can be significantly affected by variations in atmospheric, illumination or environmental conditions typically occurring within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, what propagates significant mismodeling errors throughout the whole unmixing process and compromises the quality of its results. Therefore, large efforts have been recently dedicated to mitigate the effects of spectral variability in SU. This resulted in the development of algorithms that incorporate different strategies to allow the EMs to vary within a hyperspectral image, using, for instance, sets of spectral signatures known a priori, Bayesian, parametric, or local EM models. Each of these approaches has different characteristics and underlying motivations. This paper presents a comprehensive literature review contextualizing both classic and recent approaches to solve this problem. We give a detailed evaluation of the sources of spectral variability and their effect in image spectra. Furthermore, we propose a new taxonomy that organizes existing works according to a practitioner's point of view, based on the necessary amount of supervision and on the computational cost they require. We also review methods used to construct spectral libraries (which are required by many SU techniques) based on the observed hyperspectral image, as well as algorithms for library augmentation and reduction. Finally, we conclude the paper with some discussions and an outline of possible future directions for the field.
△ Less
Submitted 6 April, 2021; v1 submitted 20 January, 2020;
originally announced January 2020.
-
Spectral Variability Aware Blind Hyperspectral Image Unmixing Based on Convex Geometry
Authors:
Lucas Drumetz,
Jocelyn Chanussot,
Christian Jutten,
Wing-Kin Ma,
Akira Iwasaki
Abstract:
Hyperspectral image unmixing has proven to be a useful technique to interpret hyperspectral data, and is a prolific research topic in the community. Most of the approaches used to perform linear unmixing are based on convex geometry concepts, because of the strong geometrical structure of the linear mixing model. However, two main phenomena lead to question this model, namely nonlinearities and th…
▽ More
Hyperspectral image unmixing has proven to be a useful technique to interpret hyperspectral data, and is a prolific research topic in the community. Most of the approaches used to perform linear unmixing are based on convex geometry concepts, because of the strong geometrical structure of the linear mixing model. However, two main phenomena lead to question this model, namely nonlinearities and the spectral variability of the materials. Many algorithms based on convex geometry are still used when considering these two limitations of the linear model. A natural question is to wonder to what extent these concepts and tools (Intrinsic Dimensionality estimation, endmember extraction algorithms, pixel purity) can be safely used in these different scenarios. In this paper, we analyze them with a focus on endmember variability, assuming that the linear model holds. In the light of this analysis, we propose an integrated unmixing chain which tries to adress the shortcomings of the classical tools used in the linear case, based on our previously proposed extended linear mixing model. We show the interest of the proposed approach on simulated and real datasets.
△ Less
Submitted 8 April, 2019;
originally announced April 2019.
-
Spectral Unmixing: A Derivation of the Extended Linear Mixing Model from the Hapke Model
Authors:
Lucas Drumetz,
Jocelyn Chanussot,
Christian Jutten
Abstract:
In hyperspectral imaging, spectral unmixing aims at decomposing the image into a set of reference spectral signatures corresponding to the materials present in the observed scene and their relative proportions in every pixel. While a linear mixing model was used for a long time, the complex nature of the physical mixing processes, led to shift the community's attention towards nonlinear models and…
▽ More
In hyperspectral imaging, spectral unmixing aims at decomposing the image into a set of reference spectral signatures corresponding to the materials present in the observed scene and their relative proportions in every pixel. While a linear mixing model was used for a long time, the complex nature of the physical mixing processes, led to shift the community's attention towards nonlinear models and algorithms accounting for the variability of the endmembers. Such intra class variations are due to local changes in the physico-chemical composition of the materials, and to illumination changes. In the physical remote sensing community, a popular model accounting for illumination variability is the radiative transfer model proposed by Hapke. It is however too complex to be directly used in hyperspectral unmixing in a tractable way. Instead, the Extended Linear Mixing Model (ELMM) allows to easily unmix hyperspectral data accounting for changing illumination conditions. In this letter, we show that the ELMM can be obtained from the Hapke model by successive simplifiying physical assumptions, thus theoretically confirming its relevance to handle illumination induced variability in the unmixing problem.
△ Less
Submitted 24 July, 2019; v1 submitted 28 March, 2019;
originally announced March 2019.
-
Optimal Measurement Times for a Small Number of Measures of a Brownian Motion over a Finite Period
Authors:
Alexandre Aksenov,
Pierre-Olivier Amblard,
Olivier Michel,
Christian Jutten
Abstract:
The measure timetable plays a critical role for the accuracy of the estimator. This article deals with the optimization of the schedule of measures for observing a random process in time using a Kalman filter, when the length of the process is finite and fixed, and a fixed number of measures are available. The measuring devices are allowed to differ. The mean variance of the estimator is chosen as…
▽ More
The measure timetable plays a critical role for the accuracy of the estimator. This article deals with the optimization of the schedule of measures for observing a random process in time using a Kalman filter, when the length of the process is finite and fixed, and a fixed number of measures are available. The measuring devices are allowed to differ. The mean variance of the estimator is chosen as criterion for optimality. The cases of $1$ or $2$ measures are studied in detail, and analytical formulas are provided.
△ Less
Submitted 16 February, 2019;
originally announced February 2019.
-
Schur's Lemma for Coupled Reducibility and Coupled Normality
Authors:
Dana Lahat,
Christian Jutten,
Helene Shapiro
Abstract:
Let $\mathcal A = \{A_{ij} \}_{i, j \in \mathcal I}$, where $\mathcal I$ is an index set, be a doubly indexed family of matrices, where $A_{ij}$ is $n_i \times n_j$. For each $i \in \mathcal I$, let $\mathcal V_i$ be an $n_i$-dimensional vector space. We say $\mathcal A$ is reducible in the coupled sense if there exist subspaces, $\mathcal U_i \subseteq \mathcal V_i$, with…
▽ More
Let $\mathcal A = \{A_{ij} \}_{i, j \in \mathcal I}$, where $\mathcal I$ is an index set, be a doubly indexed family of matrices, where $A_{ij}$ is $n_i \times n_j$. For each $i \in \mathcal I$, let $\mathcal V_i$ be an $n_i$-dimensional vector space. We say $\mathcal A$ is reducible in the coupled sense if there exist subspaces, $\mathcal U_i \subseteq \mathcal V_i$, with $\mathcal U_i \neq \{0\}$ for at least one $i \in \mathcal I$, and $\mathcal U_i \neq \mathcal V_i$ for at least one $i$, such that $A_{ij} (\mathcal U_j) \subseteq \mathcal U_i$ for all $i, j$. Let $\mathcal B = \{B_{ij} \}_{i, j \in \mathcal I}$ also be a doubly indexed family of matrices, where $B_{ij}$ is $m_i \times m_j$. For each $i \in \mathcal I$, let $X_i$ be a matrix of size $n_i \times m_i$. Suppose $A_{ij} X_j = X_i B_{ij}$ for all~$i, j$. We prove versions of Schur's Lemma for $\mathcal A, \mathcal B$ satisfying coupled irreducibility conditions. We also consider a refinement of Schur's Lemma for sets of normal matrices and prove corresponding versions for $\mathcal A, \mathcal B$ satisfying coupled normality and coupled irreducibility conditions.
△ Less
Submitted 29 November, 2018; v1 submitted 20 November, 2018;
originally announced November 2018.
-
Hyperspectral Image Unmixing with Endmember Bundles and Group Sparsity Inducing Mixed Norms
Authors:
Lucas Drumetz,
Travis R. Meyer,
Jocelyn Chanussot,
Andrea L. Bertozzi,
Christian Jutten
Abstract:
Hyperspectral images provide much more information than conventional imaging techniques, allowing a precise identification of the materials in the observed scene, but because of the limited spatial resolution, the observations are usually mixtures of the contributions of several materials. The spectral unmixing problem aims at recovering the spectra of the pure materials of the scene (endmembers),…
▽ More
Hyperspectral images provide much more information than conventional imaging techniques, allowing a precise identification of the materials in the observed scene, but because of the limited spatial resolution, the observations are usually mixtures of the contributions of several materials. The spectral unmixing problem aims at recovering the spectra of the pure materials of the scene (endmembers), along with their proportions (abundances) in each pixel. In order to deal with the intra-class variability of the materials and the induced spectral variability of the endmembers, several spectra per material, constituting endmember bundles, can be considered. However, the usual abundance estimation techniques do not take advantage of the particular structure of these bundles, organized into groups of spectra. In this paper, we propose to use group sparsity by introducing mixed norms in the abundance estimation optimization problem. In particular, we propose a new penalty which simultaneously enforces group and within group sparsity, to the cost of being nonconvex. All the proposed penalties are compatible with the abundance sum-to-one constraint, which is not the case with traditional sparse regression. We show on simulated and real datasets that well chosen penalties can significantly improve the unmixing performance compared to the naive bundle approach.
△ Less
Submitted 28 March, 2019; v1 submitted 25 May, 2018;
originally announced May 2018.
-
Dynamical spectral unmixing of multitemporal hyperspectral images
Authors:
Simon Henrot,
Jocelyn Chanussot,
Christian Jutten
Abstract:
In this paper, we consider the problem of unmixing a time series of hyperspectral images. We propose a dynamical model based on linear mixing processes at each time instant. The spectral signatures and fractional abundances of the pure materials in the scene are seen as latent variables, and assumed to follow a general dynamical structure. Based on a simplified version of this model, we derive an…
▽ More
In this paper, we consider the problem of unmixing a time series of hyperspectral images. We propose a dynamical model based on linear mixing processes at each time instant. The spectral signatures and fractional abundances of the pure materials in the scene are seen as latent variables, and assumed to follow a general dynamical structure. Based on a simplified version of this model, we derive an efficient spectral unmixing algorithm to estimate the latent variables by performing alternating minimizations. The performance of the proposed approach is demonstrated on synthetic and real multitemporal hyperspectral images.
△ Less
Submitted 14 October, 2015;
originally announced October 2015.
-
Sparse Channel Estimation by Factor Graphs
Authors:
Rad Niazadeh,
Masoud Babaie-Zadeh,
Christian Jutten
Abstract:
The problem of estimating a sparse channel, i.e. a channel with a few non-zero taps, appears in various areas of communications. Recently, we have developed an algorithm based on iterative alternating minimization which iteratively detects the location and the value of the taps. This algorithms involves an approximate Maximum A Posteriori (MAP) probability scheme for detection of the location of t…
▽ More
The problem of estimating a sparse channel, i.e. a channel with a few non-zero taps, appears in various areas of communications. Recently, we have developed an algorithm based on iterative alternating minimization which iteratively detects the location and the value of the taps. This algorithms involves an approximate Maximum A Posteriori (MAP) probability scheme for detection of the location of taps, while a least square method is used for estimating the values at each iteration. In this work, based on the method of factor graphs and message passing algorithms, we will compute an exact solution for the MAP estimation problem. Indeed, we first find a factor graph model of this problem, and then perform the well-known min-sum algorithm on the edges of this graph. Consequently, we will find an exact estimator for the MAP problem that its complexity grows linearly with respect to the channel memory. By substituting this estimator in the mentioned alternating minimization method, we will propose an estimator that will nearly achieve the Cramer-Rao bound of the genie-aided estimation of sparse channels (estimation based on knowing the location of non-zero taps of the channel), while it can perform faster than most of the proposed algorithms in literature.
△ Less
Submitted 26 August, 2013;
originally announced August 2013.
-
Recovery of Low-Rank Matrices under Affine Constraints via a Smoothed Rank Function
Authors:
Mohammadreza Malek-Mohammadi,
Massoud Babaie-Zadeh,
Arash Amini,
Christian Jutten
Abstract:
In this paper, the problem of matrix rank minimization under affine constraints is addressed. The state-of-the-art algorithms can recover matrices with a rank much less than what is sufficient for the uniqueness of the solution of this optimization problem. We propose an algorithm based on a smooth approximation of the rank function, which practically improves recovery limits on the rank of the so…
▽ More
In this paper, the problem of matrix rank minimization under affine constraints is addressed. The state-of-the-art algorithms can recover matrices with a rank much less than what is sufficient for the uniqueness of the solution of this optimization problem. We propose an algorithm based on a smooth approximation of the rank function, which practically improves recovery limits on the rank of the solution. This approximation leads to a non-convex program; thus, to avoid getting trapped in local solutions, we use the following scheme. Initially, a rough approximation of the rank function subject to the affine constraints is optimized. As the algorithm proceeds, finer approximations of the rank are optimized and the solver is initialized with the solution of the previous approximation until reaching the desired accuracy.
On the theoretical side, benefiting from the spherical section property, we will show that the sequence of the solutions of the approximating function converges to the minimum rank solution. On the experimental side, it will be shown that the proposed algorithm, termed SRF standing for Smoothed Rank Function, can recover matrices which are unique solutions of the rank minimization problem and yet not recoverable by nuclear norm minimization. Furthermore, it will be demonstrated that, in completing partially observed matrices, the accuracy of SRF is considerably and consistently better than some famous algorithms when the number of revealed entries is close to the minimum number of parameters that uniquely represent a low-rank matrix.
△ Less
Submitted 26 December, 2013; v1 submitted 10 August, 2013;
originally announced August 2013.
-
On the error of estimating the sparsest solution of underdetermined linear systems
Authors:
Massoud Babaie-Zadeh,
Christian Jutten,
Hosein Mohimani
Abstract:
Let A be an n by m matrix with m>n, and suppose that the underdetermined linear system As=x admits a sparse solution s0 for which ||s0||_0 < 1/2 spark(A). Such a sparse solution is unique due to a well-known uniqueness theorem. Suppose now that we have somehow a solution s_hat as an estimation of s0, and suppose that s_hat is only `approximately sparse', that is, many of its components are very sm…
▽ More
Let A be an n by m matrix with m>n, and suppose that the underdetermined linear system As=x admits a sparse solution s0 for which ||s0||_0 < 1/2 spark(A). Such a sparse solution is unique due to a well-known uniqueness theorem. Suppose now that we have somehow a solution s_hat as an estimation of s0, and suppose that s_hat is only `approximately sparse', that is, many of its components are very small and nearly zero, but not mathematically equal to zero. Is such a solution necessarily close to the true sparsest solution? More generally, is it possible to construct an upper bound on the estimation error ||s_hat-s0||_2 without knowing s0? The answer is positive, and in this paper we construct such a bound based on minimal singular values of submatrices of A. We will also state a tight bound, which is more complicated, but besides being tight, enables us to study the case of random dictionaries and obtain probabilistic upper bounds. We will also study the noisy case, that is, where x=As+n. Moreover, we will see that where ||s0||_0 grows, to obtain a predetermined guaranty on the maximum of ||s_hat-s0||_2, s_hat is needed to be sparse with a better approximation. This can be seen as an explanation to the fact that the estimation quality of sparse recovery algorithms degrades where ||s0||_0 grows.
△ Less
Submitted 4 December, 2011;
originally announced December 2011.
-
Fast Sparse Decomposition by Iterative Detection-Estimation
Authors:
Arash Ali Amini,
Massoud Babaie-Zadeh,
Christian Jutten
Abstract:
Finding sparse solutions of underdetermined systems of linear equations is a fundamental problem in signal processing and statistics which has become a subject of interest in recent years. In general, these systems have infinitely many solutions. However, it may be shown that sufficiently sparse solutions may be identified uniquely. In other words, the corresponding linear transformation will be i…
▽ More
Finding sparse solutions of underdetermined systems of linear equations is a fundamental problem in signal processing and statistics which has become a subject of interest in recent years. In general, these systems have infinitely many solutions. However, it may be shown that sufficiently sparse solutions may be identified uniquely. In other words, the corresponding linear transformation will be invertible if we restrict its domain to sufficiently sparse vectors. This property may be used, for example, to solve the underdetermined Blind Source Separation (BSS) problem, or to find sparse representation of a signal in an `overcomplete' dictionary of primitive elements (i.e., the so-called atomic decomposition). The main drawback of current methods of finding sparse solutions is their computational complexity. In this paper, we will show that by detecting `active' components of the (potential) solution, i.e., those components having a considerable value, a framework for fast solution of the problem may be devised. The idea leads to a family of algorithms, called `Iterative Detection-Estimation (IDE)', which converge to the solution by successive detection and estimation of its active part. Comparing the performance of IDE(s) with one of the most successful method to date, which is based on Linear Programming (LP), an improvement in speed of about two to three orders of magnitude is observed.
△ Less
Submitted 20 September, 2010;
originally announced September 2010.
-
Bayesian Hypothesis Testing for Sparse Representation
Authors:
Hadi Zayyani,
Massoud Babaie-Zadeh,
Christian Jutten
Abstract:
In this paper, we propose a Bayesian Hypothesis Testing Algorithm (BHTA) for sparse representation. It uses the Bayesian framework to determine active atoms in sparse representation of a signal.
The Bayesian hypothesis testing based on three assumptions, determines the active atoms from the correlations and leads to the activity measure as proposed in Iterative Detection Estimation (IDE) algorit…
▽ More
In this paper, we propose a Bayesian Hypothesis Testing Algorithm (BHTA) for sparse representation. It uses the Bayesian framework to determine active atoms in sparse representation of a signal.
The Bayesian hypothesis testing based on three assumptions, determines the active atoms from the correlations and leads to the activity measure as proposed in Iterative Detection Estimation (IDE) algorithm. In fact, IDE uses an arbitrary decreasing sequence of thresholds while the proposed algorithm is based on a sequence which derived from hypothesis testing. So, Bayesian hypothesis testing framework leads to an improved version of the IDE algorithm.
The simulations show that Hard-version of our suggested algorithm achieves one of the best results in terms of estimation accuracy among the algorithms which have been implemented in our simulations, while it has the greatest complexity in terms of simulation time.
△ Less
Submitted 21 August, 2010;
originally announced August 2010.
-
On the Achievability of Cramér-Rao Bound In Noisy Compressed Sensing
Authors:
Rad Niazadeh,
Masoud Babaie-Zadeh,
Christian Jutten
Abstract:
Recently, it has been proved in Babadi et al. that in noisy compressed sensing, a joint typical estimator can asymptotically achieve the Cramer-Rao lower bound of the problem.To prove this result, this paper used a lemma,which is provided in Akcakaya et al,that comprises the main building block of the proof. This lemma is based on the assumption of Gaussianity of the measurement matrix and its ran…
▽ More
Recently, it has been proved in Babadi et al. that in noisy compressed sensing, a joint typical estimator can asymptotically achieve the Cramer-Rao lower bound of the problem.To prove this result, this paper used a lemma,which is provided in Akcakaya et al,that comprises the main building block of the proof. This lemma is based on the assumption of Gaussianity of the measurement matrix and its randomness in the domain of noise. In this correspondence, we generalize the results obtained in Babadi et al by drop** the Gaussianity assumption on the measurement matrix. In fact, by considering the measurement matrix as a deterministic matrix in our analysis, we find a theorem similar to the main theorem of Babadi et al for a family of randomly generated (but deterministic in the noise domain) measurement matrices that satisfy a generalized condition known as The Concentration of Measures Inequality. By this, we finally show that under our generalized assumptions, the Cramer-Rao bound of the estimation is achievable by using the typical estimator introduced in Babadi et al.
△ Less
Submitted 25 August, 2013; v1 submitted 13 June, 2010;
originally announced June 2010.
-
On the stable recovery of the sparsest overcomplete representations in presence of noise
Authors:
Massoud Babaie-Zadeh,
Christian Jutten
Abstract:
Let x be a signal to be sparsely decomposed over a redundant dictionary A, i.e., a sparse coefficient vector s has to be found such that x=As. It is known that this problem is inherently unstable against noise, and to overcome this instability, the authors of [Stable Recovery; Donoho et.al., 2006] have proposed to use an "approximate" decomposition, that is, a decomposition satisfying ||x - A s||…
▽ More
Let x be a signal to be sparsely decomposed over a redundant dictionary A, i.e., a sparse coefficient vector s has to be found such that x=As. It is known that this problem is inherently unstable against noise, and to overcome this instability, the authors of [Stable Recovery; Donoho et.al., 2006] have proposed to use an "approximate" decomposition, that is, a decomposition satisfying ||x - A s|| < δ, rather than satisfying the exact equality x = As. Then, they have shown that if there is a decomposition with ||s||_0 < (1+M^{-1})/2, where M denotes the coherence of the dictionary, this decomposition would be stable against noise. On the other hand, it is known that a sparse decomposition with ||s||_0 < spark(A)/2 is unique. In other words, although a decomposition with ||s||_0 < spark(A)/2 is unique, its stability against noise has been proved only for highly more restrictive decompositions satisfying ||s||_0 < (1+M^{-1})/2, because usually (1+M^{-1})/2 << spark(A)/2.
This limitation maybe had not been very important before, because ||s||_0 < (1+M^{-1})/2 is also the bound which guaranties that the sparse decomposition can be found via minimizing the L1 norm, a classic approach for sparse decomposition. However, with the availability of new algorithms for sparse decomposition, namely SL0 and Robust-SL0, it would be important to know whether or not unique sparse decompositions with (1+M^{-1})/2 < ||s||_0 < spark(A)/2 are stable. In this paper, we show that such decompositions are indeed stable. In other words, we extend the stability bound from ||s||_0 < (1+M^{-1})/2 to the whole uniqueness range ||s||_0 < spark(A)/2. In summary, we show that "all unique sparse decompositions are stably recoverable". Moreover, we see that sparser decompositions are "more stable".
△ Less
Submitted 2 June, 2010;
originally announced June 2010.
-
Bayesian Cramér-Rao Bound for Noisy Non-Blind and Blind Compressed Sensing
Authors:
Hadi Zayyani,
Massoud Babaie-Zadeh,
Christian Jutten
Abstract:
In this paper, we address the theoretical limitations in reconstructing sparse signals (in a known complete basis) using compressed sensing framework. We also divide the CS to non-blind and blind cases. Then, we compute the Bayesian Cramer-Rao bound for estimating the sparse coefficients while the measurement matrix elements are independent zero mean random variables. Simulation results show a lar…
▽ More
In this paper, we address the theoretical limitations in reconstructing sparse signals (in a known complete basis) using compressed sensing framework. We also divide the CS to non-blind and blind cases. Then, we compute the Bayesian Cramer-Rao bound for estimating the sparse coefficients while the measurement matrix elements are independent zero mean random variables. Simulation results show a large gap between the lower bound and the performance of the practical algorithms when the number of measurements are low.
△ Less
Submitted 24 May, 2010;
originally announced May 2010.
-
Sparse Recovery using Smoothed $\ell^0$ (SL0): Convergence Analysis
Authors:
Hosein Mohimani,
Massoud Babaie-Zadeh,
Irina Gorodnitsky,
Christian Jutten
Abstract:
Finding the sparse solution of an underdetermined system of linear equations has many applications, especially, it is used in Compressed Sensing (CS), Sparse Component Analysis (SCA), and sparse decomposition of signals on overcomplete dictionaries. We have recently proposed a fast algorithm, called Smoothed $\ell^0$ (SL0), for this task. Contrary to many other sparse recovery algorithms, SL0 is…
▽ More
Finding the sparse solution of an underdetermined system of linear equations has many applications, especially, it is used in Compressed Sensing (CS), Sparse Component Analysis (SCA), and sparse decomposition of signals on overcomplete dictionaries. We have recently proposed a fast algorithm, called Smoothed $\ell^0$ (SL0), for this task. Contrary to many other sparse recovery algorithms, SL0 is not based on minimizing the $\ell^1$ norm, but it tries to directly minimize the $\ell^0$ norm of the solution. The basic idea of SL0 is optimizing a sequence of certain (continuous) cost functions approximating the $\ell^0$ norm of a vector. However, in previous papers, we did not provide a complete convergence proof for SL0. In this paper, we study the convergence properties of SL0, and show that under a certain sparsity constraint in terms of Asymmetric Restricted Isometry Property (ARIP), and with a certain choice of parameters, the convergence of SL0 to the sparsest solution is guaranteed. Moreover, we study the complexity of SL0, and we show that whenever the dimension of the dictionary grows, the complexity of SL0 increases with the same order as Matching Pursuit (MP), which is one of the fastest existing sparse recovery methods, while contrary to MP, its convergence to the sparsest solution is guaranteed under certain conditions which are satisfied through the choice of parameters.
△ Less
Submitted 27 January, 2010;
originally announced January 2010.
-
A New Trend in Optimization on Multi Overcomplete Dictionary toward Inpainting
Authors:
SeyyedMajid Valiollahzadeh,
Mohammad Nazari,
Massoud Babaie-Zadeh,
Christian Jutten
Abstract:
Recently, great attention was intended toward overcomplete dictionaries and the sparse representations they can provide. In a wide variety of signal processing problems, sparsity serves a crucial property leading to high performance. Inpainting, the process of reconstructing lost or deteriorated parts of images or videos, is an interesting application which can be handled by suitably decompositi…
▽ More
Recently, great attention was intended toward overcomplete dictionaries and the sparse representations they can provide. In a wide variety of signal processing problems, sparsity serves a crucial property leading to high performance. Inpainting, the process of reconstructing lost or deteriorated parts of images or videos, is an interesting application which can be handled by suitably decomposition of an image through combination of overcomplete dictionaries. This paper addresses a novel technique of such a decomposition and investigate that through inpainting of images. Simulations are presented to demonstrate the validation of our approach.
△ Less
Submitted 12 December, 2008;
originally announced December 2008.
-
On the blind source separation of human electroencephalogram by approximate joint diagonalization of second order statistics
Authors:
Marco Congedo,
Cédric Gouy-Pailler,
Christian Jutten
Abstract:
Over the last ten years blind source separation (BSS) has become a prominent processing tool in the study of human electroencephalography (EEG). Without relying on head modeling BSS aims at estimating both the waveform and the scalp spatial pattern of the intracranial dipolar current responsible of the observed EEG. In this review we begin by placing the BSS linear instantaneous model of EEG wit…
▽ More
Over the last ten years blind source separation (BSS) has become a prominent processing tool in the study of human electroencephalography (EEG). Without relying on head modeling BSS aims at estimating both the waveform and the scalp spatial pattern of the intracranial dipolar current responsible of the observed EEG. In this review we begin by placing the BSS linear instantaneous model of EEG within the framework of brain volume conduction theory. We then review the concept and current practice of BSS based on second-order statistics (SOS) and on higher-order statistics (HOS), the latter better known as independent component analysis (ICA). Using neurophysiological knowledge we consider the fitness of SOS-based and HOS-based methods for the extraction of spontaneous and induced EEG and their separation from extra-cranial artifacts. We then illustrate a general BSS scheme operating in the time-frequency domain using SOS only. The scheme readily extends to further data expansions in order to capture experimental source of variations as well. A simple and efficient implementation based on the approximate joint diagonalization of Fourier cospectral matrices is described (AJDC). We conclude discussing useful aspects of BSS analysis of EEG, including its assumptions and limitations.
△ Less
Submitted 2 December, 2008;
originally announced December 2008.
-
Approximate Sparse Decomposition Based on Smoothed L0-Norm
Authors:
Hamed Firouzi,
Masoud Farivar,
Massoud Babaie-Zadeh,
Christian Jutten
Abstract:
In this paper, we propose a method to address the problem of source estimation for Sparse Component Analysis (SCA) in the presence of additive noise. Our method is a generalization of a recently proposed method (SL0), which has the advantage of directly minimizing the L0-norm instead of L1-norm, while being very fast. SL0 is based on minimization of the smoothed L0-norm subject to As=x. In order…
▽ More
In this paper, we propose a method to address the problem of source estimation for Sparse Component Analysis (SCA) in the presence of additive noise. Our method is a generalization of a recently proposed method (SL0), which has the advantage of directly minimizing the L0-norm instead of L1-norm, while being very fast. SL0 is based on minimization of the smoothed L0-norm subject to As=x. In order to better estimate the source vector for noisy mixtures, we suggest then to remove the constraint As=x, by relaxing exact equality to an approximation (we call our method Smoothed L0-norm Denoising or SL0DN). The final result can then be obtained by minimization of a proper linear combination of the smoothed L0-norm and a cost function for the approximation. Experimental results emphasize on the significant enhancement of the modified method in noisy cases.
△ Less
Submitted 18 November, 2008;
originally announced November 2008.
-
A First Step to Convolutive Sparse Representation
Authors:
Hamed Firouzi,
Massoud Babaie-Zadeh,
Aria Ghasemian,
Christian Jutten
Abstract:
In this paper an extension of the sparse decomposition problem is considered and an algorithm for solving it is presented. In this extension, it is known that one of the shifted versions of a signal s (not necessarily the original signal itself) has a sparse representation on an overcomplete dictionary, and we are looking for the sparsest representation among the representations of all the shift…
▽ More
In this paper an extension of the sparse decomposition problem is considered and an algorithm for solving it is presented. In this extension, it is known that one of the shifted versions of a signal s (not necessarily the original signal itself) has a sparse representation on an overcomplete dictionary, and we are looking for the sparsest representation among the representations of all the shifted versions of s. Then, the proposed algorithm finds simultaneously the amount of the required shift, and the sparse representation. Experimental results emphasize on the performance of our algorithm.
△ Less
Submitted 20 September, 2008;
originally announced September 2008.
-
A fast approach for overcomplete sparse decomposition based on smoothed L0 norm
Authors:
Hossein Mohimani,
Massoud Babaie-Zadeh,
Christian Jutten
Abstract:
In this paper, a fast algorithm for overcomplete sparse decomposition, called SL0, is proposed. The algorithm is essentially a method for obtaining sparse solutions of underdetermined systems of linear equations, and its applications include underdetermined Sparse Component Analysis (SCA), atomic decomposition on overcomplete dictionaries, compressed sensing, and decoding real field codes. Contr…
▽ More
In this paper, a fast algorithm for overcomplete sparse decomposition, called SL0, is proposed. The algorithm is essentially a method for obtaining sparse solutions of underdetermined systems of linear equations, and its applications include underdetermined Sparse Component Analysis (SCA), atomic decomposition on overcomplete dictionaries, compressed sensing, and decoding real field codes. Contrary to previous methods, which usually solve this problem by minimizing the L1 norm using Linear Programming (LP) techniques, our algorithm tries to directly minimize the L0 norm. It is experimentally shown that the proposed algorithm is about two to three orders of magnitude faster than the state-of-the-art interior-point LP solvers, while providing the same (or better) accuracy.
△ Less
Submitted 16 September, 2008; v1 submitted 15 September, 2008;
originally announced September 2008.
