Showing 1–29 of 29 results for author: Pascal, F

Random matrix theory improved Fréchet mean of symmetric positive definite matrices

Authors: Florent Bouchard, Ammar Mian, Malik Tiomoko, Guillaume Ginolhac, Frédéric Pascal

Abstract: In this study, we consider the realm of covariance matrices in machine learning, particularly focusing on computing Fréchet means on the manifold of symmetric positive definite matrices, commonly referred to as Karcher or geometric means. Such means are leveraged in numerous machine-learning tasks. Relying on advanced statistical tools, we introduce a random matrix theory-based method that estimat… ▽ More In this study, we consider the realm of covariance matrices in machine learning, particularly focusing on computing Fréchet means on the manifold of symmetric positive definite matrices, commonly referred to as Karcher or geometric means. Such means are leveraged in numerous machine-learning tasks. Relying on advanced statistical tools, we introduce a random matrix theory-based method that estimates Fréchet means, which is particularly beneficial when dealing with low sample support and a high number of matrices to average. Our experimental evaluation, involving both synthetic and real-world EEG and hyperspectral datasets, shows that we largely outperform state-of-the-art methods. △ Less

Submitted 5 June, 2024; v1 submitted 10 May, 2024; originally announced May 2024.

On Elliptical and Inverse Elliptical Wishart distributions: Review, new results, and applications

Authors: Imen Ayadi, Florent Bouchard, Frédéric Pascal

Abstract: This paper deals with matrix-variate distributions, from Wishart to Inverse Elliptical Wishart distributions over the set of symmetric definite positive matrices. Similar to the multivariate scenario, (Inverse) Elliptical Wishart distributions form a vast and general family of distributions, encompassing, for instance, Wishart or $t$-Wishart ones. The first objective of this study is to present a… ▽ More This paper deals with matrix-variate distributions, from Wishart to Inverse Elliptical Wishart distributions over the set of symmetric definite positive matrices. Similar to the multivariate scenario, (Inverse) Elliptical Wishart distributions form a vast and general family of distributions, encompassing, for instance, Wishart or $t$-Wishart ones. The first objective of this study is to present a unified overview of Wishart, Inverse Wishart, Elliptical Wishart, and Inverse Elliptical Wishart distributions through their fundamental properties. This involves leveraging the stochastic representation of these distributions to establish key statistical properties of the Normalized Wishart distribution. Subsequently, this enables the computation of expectations, variances, and Kronecker moments for Elliptical Wishart and Inverse Elliptical Wishart distributions. As an illustrative application, the practical utility of these generalized Elliptical Wishart distributions is demonstrated using a real electroencephalographic dataset. This showcases their effectiveness in accurately modeling heterogeneous data. △ Less

Submitted 26 April, 2024; originally announced April 2024.

Sparse PCA with False Discovery Rate Controlled Variable Selection

Authors: Jasin Machkour, Arnaud Breloy, Michael Muma, Daniel P. Palomar, Frédéric Pascal

Abstract: Sparse principal component analysis (PCA) aims at map** large dimensional data to a linear subspace of lower dimension. By imposing loading vectors to be sparse, it performs the double duty of dimension reduction and variable selection. Sparse PCA algorithms are usually expressed as a trade-off between explained variance and sparsity of the loading vectors (i.e., number of selected variables). A… ▽ More Sparse principal component analysis (PCA) aims at map** large dimensional data to a linear subspace of lower dimension. By imposing loading vectors to be sparse, it performs the double duty of dimension reduction and variable selection. Sparse PCA algorithms are usually expressed as a trade-off between explained variance and sparsity of the loading vectors (i.e., number of selected variables). As a high explained variance is not necessarily synonymous with relevant information, these methods are prone to select irrelevant variables. To overcome this issue, we propose an alternative formulation of sparse PCA driven by the false discovery rate (FDR). We then leverage the Terminating-Random Experiments (T-Rex) selector to automatically determine an FDR-controlled support of the loading vectors. A major advantage of the resulting T-Rex PCA is that no sparsity parameter tuning is required. Numerical experiments and a stock market data example demonstrate a significant performance improvement. △ Less

Submitted 16 January, 2024; originally announced January 2024.

Comments: Published in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), scheduled for 14-19 April 2024 in Seoul, Korea

Deinterleaving RADAR emitters with optimal transport distances

Authors: Manon Mottier, Gilles chardon, Frédéric Pascal

Abstract: Detection and identification of emitters provide vital information for defensive strategies in electronic intelligence. Based on a received signal containing pulses from an unknown number of emitters, this paper introduces an unsupervised methodology for deinterleaving RADAR signals based on a combination of clustering algorithms and optimal transport distances. The first step involves separating… ▽ More Detection and identification of emitters provide vital information for defensive strategies in electronic intelligence. Based on a received signal containing pulses from an unknown number of emitters, this paper introduces an unsupervised methodology for deinterleaving RADAR signals based on a combination of clustering algorithms and optimal transport distances. The first step involves separating the pulses with a clustering algorithm under the constraint that the pulses of two different emitters cannot belong to the same cluster. Then, as the emitters exhibit complex behavior and can be represented by several clusters, we propose a hierarchical clustering algorithm based on an optimal transport distance to merge these clusters. A variant is also developed, capable of handling more complex signals. Finally, the proposed methodology is evaluated on simulated data provided through a realistic simulator. Results show that the proposed methods are capable of deinterleaving complex RADAR signals. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Convex Parameter Estimation of Perturbed Multivariate Generalized Gaussian Distributions

Authors: Nora Ouzir, Frédéric Pascal, Jean-Christophe Pesquet

Abstract: The multivariate generalized Gaussian distribution (MGGD), also known as the multivariate exponential power (MEP) distribution, is widely used in signal and image processing. However, estimating MGGD parameters, which is required in practical applications, still faces specific theoretical challenges. In particular, establishing convergence properties for the standard fixed-point approach when both… ▽ More The multivariate generalized Gaussian distribution (MGGD), also known as the multivariate exponential power (MEP) distribution, is widely used in signal and image processing. However, estimating MGGD parameters, which is required in practical applications, still faces specific theoretical challenges. In particular, establishing convergence properties for the standard fixed-point approach when both the distribution mean and the scatter (or the precision) matrix are unknown is still an open problem. In robust estimation, imposing classical constraints on the precision matrix, such as sparsity, has been limited by the non-convexity of the resulting cost function. This paper tackles these issues from an optimization viewpoint by proposing a convex formulation with well-established convergence properties. We embed our analysis in a noisy scenario where robustness is induced by modelling multiplicative perturbations. The resulting framework is flexible as it combines a variety of regularizations for the precision matrix, the mean and model perturbations. This paper presents proof of the desired theoretical properties, specifies the conditions preserving these properties for different regularization choices and designs a general proximal primal-dual optimization strategy. The experiments show a more accurate precision and covariance matrix estimation with similar performance for the mean vector parameter compared to Tyler's M-estimator. In a high-dimensional setting, the proposed method outperforms the classical GLASSO, one of its robust extensions, and the regularized Tyler's estimator. △ Less

Submitted 12 December, 2023; originally announced December 2023.

Choosing the parameter of the Fermat distance: navigating geometry and noise

Authors: Frédéric Chazal, Laure Ferraris, Pablo Groisman, Matthieu Jonckheere, Frédéric Pascal, Facundo Sapienza

Abstract: The Fermat distance has been recently established as a useful tool for machine learning tasks when a natural distance is not directly available to the practitioner or to improve the results given by Euclidean distances by exploding the geometrical and statistical properties of the dataset. This distance depends on a parameter $α$ that greatly impacts the performance of subsequent tasks. Ideally, t… ▽ More The Fermat distance has been recently established as a useful tool for machine learning tasks when a natural distance is not directly available to the practitioner or to improve the results given by Euclidean distances by exploding the geometrical and statistical properties of the dataset. This distance depends on a parameter $α$ that greatly impacts the performance of subsequent tasks. Ideally, the value of $α$ should be large enough to navigate the geometric intricacies inherent to the problem. At the same, it should remain restrained enough to sidestep any deleterious ramifications stemming from noise during the process of distance estimation. We study both theoretically and through simulations how to select this parameter. △ Less

Submitted 30 November, 2023; originally announced November 2023.

FEMDA: a unified framework for discriminant analysis

Authors: Pierre Houdouin, Matthieu Jonckheere, Frederic Pascal

Abstract: Although linear and quadratic discriminant analysis are widely recognized classical methods, they can encounter significant challenges when dealing with non-Gaussian distributions or contaminated datasets. This is primarily due to their reliance on the Gaussian assumption, which lacks robustness. We first explain and review the classical methods to address this limitation and then present a novel… ▽ More Although linear and quadratic discriminant analysis are widely recognized classical methods, they can encounter significant challenges when dealing with non-Gaussian distributions or contaminated datasets. This is primarily due to their reliance on the Gaussian assumption, which lacks robustness. We first explain and review the classical methods to address this limitation and then present a novel approach that overcomes these issues. In this new approach, the model considered is an arbitrary Elliptically Symmetrical (ES) distribution per cluster with its own arbitrary scale parameter. This flexible model allows for potentially diverse and independent samples that may not follow identical distributions. By deriving a new decision rule, we demonstrate that maximum-likelihood parameter estimation and classification are simple, efficient, and robust compared to state-of-the-art methods. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Algorithme EM régularisé

Authors: Pierre Houdouin, Matthieu Jonkcheere, Frederic Pascal

Abstract: Expectation-Maximization (EM) algorithm is a widely used iterative algorithm for computing maximum likelihood estimate when dealing with Gaussian Mixture Model (GMM). When the sample size is smaller than the data dimension, this could lead to a singular or poorly conditioned covariance matrix and, thus, to performance reduction. This paper presents a regularized version of the EM algorithm that ef… ▽ More Expectation-Maximization (EM) algorithm is a widely used iterative algorithm for computing maximum likelihood estimate when dealing with Gaussian Mixture Model (GMM). When the sample size is smaller than the data dimension, this could lead to a singular or poorly conditioned covariance matrix and, thus, to performance reduction. This paper presents a regularized version of the EM algorithm that efficiently uses prior knowledge to cope with a small sample size. This method aims to maximize a penalized GMM likelihood where regularized estimation may ensure positive definiteness of covariance matrix updates by shrinking the estimators towards some structured target covariance matrices. Finally, experiments on real data highlight the good performance of the proposed algorithm for clustering purposes △ Less

Submitted 4 July, 2023; originally announced July 2023.

Comments: in French language

FEMDA: Une méthode de classification robuste et flexible

Authors: Pierre Houdouin, Matthieu Jonckheere, Frederic Pascal

Abstract: Linear and Quadratic Discriminant Analysis (LDA and QDA) are well-known classical methods but can heavily suffer from non-Gaussian distributions and/or contaminated datasets, mainly because of the underlying Gaussian assumption that is not robust. This paper studies the robustness to scale changes in the data of a new discriminant analysis technique where each data point is drawn by its own arbitr… ▽ More Linear and Quadratic Discriminant Analysis (LDA and QDA) are well-known classical methods but can heavily suffer from non-Gaussian distributions and/or contaminated datasets, mainly because of the underlying Gaussian assumption that is not robust. This paper studies the robustness to scale changes in the data of a new discriminant analysis technique where each data point is drawn by its own arbitrary Elliptically Symmetrical (ES) distribution and its own arbitrary scale parameter. Such a model allows for possibly very heterogeneous, independent but non-identically distributed samples. The new decision rule derived is simple, fast, and robust to scale changes in the data compared to other state-of-the-art method △ Less

Submitted 4 July, 2023; originally announced July 2023.

Comments: in French language

Affine equivariant Tyler's M-estimator applied to tail parameter learning of elliptical distributions

Authors: Esa Ollila, Daniel P. Palomar, Frederic Pascal

Abstract: We propose estimating the scale parameter (mean of the eigenvalues) of the scatter matrix of an unspecified elliptically symmetric distribution using weights obtained by solving Tyler's M-estimator of the scatter matrix. The proposed Tyler's weights-based estimate (TWE) of scale is then used to construct an affine equivariant Tyler's M-estimator as a weighted sample covariance matrix using normali… ▽ More We propose estimating the scale parameter (mean of the eigenvalues) of the scatter matrix of an unspecified elliptically symmetric distribution using weights obtained by solving Tyler's M-estimator of the scatter matrix. The proposed Tyler's weights-based estimate (TWE) of scale is then used to construct an affine equivariant Tyler's M-estimator as a weighted sample covariance matrix using normalized Tyler's weights. We then develop a unified framework for estimating the unknown tail parameter of the elliptical distribution (such as the degrees of freedom (d.o.f.) $ν$ of the multivariate $t$ (MVT) distribution). Using the proposed TWE of scale, a new robust estimate of the d.o.f. parameter of MVT distribution is proposed with excellent performance in heavy-tailed scenarios, outperforming other competing methods. R-package is available that implements the proposed method. △ Less

Submitted 7 May, 2023; originally announced May 2023.

Regularized EM algorithm

Authors: Pierre Houdouin, Esa Ollila, Frederic Pascal

Abstract: Expectation-Maximization (EM) algorithm is a widely used iterative algorithm for computing (local) maximum likelihood estimate (MLE). It can be used in an extensive range of problems, including the clustering of data based on the Gaussian mixture model (GMM). Numerical instability and convergence problems may arise in situations where the sample size is not much larger than the data dimensionality… ▽ More Expectation-Maximization (EM) algorithm is a widely used iterative algorithm for computing (local) maximum likelihood estimate (MLE). It can be used in an extensive range of problems, including the clustering of data based on the Gaussian mixture model (GMM). Numerical instability and convergence problems may arise in situations where the sample size is not much larger than the data dimensionality. In such low sample support (LSS) settings, the covariance matrix update in the EM-GMM algorithm may become singular or poorly conditioned, causing the algorithm to crash. On the other hand, in many signal processing problems, a priori information can be available indicating certain structures for different cluster covariance matrices. In this paper, we present a regularized EM algorithm for GMM-s that can make efficient use of such prior knowledge as well as cope with LSS situations. The method aims to maximize a penalized GMM likelihood where regularized estimation may be used to ensure positive definiteness of covariance matrix updates and shrink the estimators towards some structured target covariance matrices. We show that the theoretical guarantees of convergence hold, leading to better performing EM algorithm for structured covariance matrix models or with low sample settings. △ Less

Submitted 27 March, 2023; originally announced March 2023.

Comments: ICASSP Conference, 4 pages, 8 figures

A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data

Authors: Florian Mouret, Alexandre Hippert-Ferrer, Frédéric Pascal, Jean-Yves Tourneret

Abstract: This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A classical imputation method, the Expectation Maximization (EM) algorithm for Gaussian mixture models, has shown interesting properties when compared to other popular approaches such as those based on k-nearest neighbors or on multiple imputations by chained equations. However, Gaussian mixture models are k… ▽ More This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A classical imputation method, the Expectation Maximization (EM) algorithm for Gaussian mixture models, has shown interesting properties when compared to other popular approaches such as those based on k-nearest neighbors or on multiple imputations by chained equations. However, Gaussian mixture models are known to be non-robust to heterogeneous data, which can lead to poor estimation performance when the data is contaminated by outliers or follows non-Gaussian distributions. To overcome this issue, a new EM algorithm is investigated for mixtures of elliptical distributions with the property of handling potential missing data. This paper shows that this problem reduces to the estimation of a mixture of Angular Gaussian distributions under generic assumptions (i.e., each sample is drawn from a mixture of elliptical distributions, which is possibly different for one sample to another). In that case, the complete-data likelihood associated with mixtures of elliptical distributions is well adapted to the EM framework with missing data thanks to its conditional distribution, which is shown to be a multivariate $t$-distribution. Experimental results on synthetic data demonstrate that the proposed algorithm is robust to outliers and can be used with non-Gaussian data. Furthermore, experiments conducted on real-world datasets show that this algorithm is very competitive when compared to other classical imputation methods. △ Less

Submitted 22 May, 2023; v1 submitted 28 January, 2022; originally announced January 2022.

Journal ref: IEEE Transactions on Signal Processing, vol. 71, pp. 1669-1682, 2023

Robust classification with flexible discriminant analysis in heterogeneous data

Authors: Pierre Houdouin, Frédéric Pascal, Matthieu Jonckheere, Andrew Wang

Abstract: Linear and Quadratic Discriminant Analysis are well-known classical methods but can heavily suffer from non-Gaussian distributions and/or contaminated datasets, mainly because of the underlying Gaussian assumption that is not robust. To fill this gap, this paper presents a new robust discriminant analysis where each data point is drawn by its own arbitrary Elliptically Symmetrical (ES) distributio… ▽ More Linear and Quadratic Discriminant Analysis are well-known classical methods but can heavily suffer from non-Gaussian distributions and/or contaminated datasets, mainly because of the underlying Gaussian assumption that is not robust. To fill this gap, this paper presents a new robust discriminant analysis where each data point is drawn by its own arbitrary Elliptically Symmetrical (ES) distribution and its own arbitrary scale parameter. Such a model allows for possibly very heterogeneous, independent but non-identically distributed samples. After deriving a new decision rule, it is shown that maximum-likelihood parameter estimation and classification are very simple, fast and robust compared to state-of-the-art methods. △ Less

Submitted 9 January, 2022; originally announced January 2022.

Comments: ICASSP conference paper, 5 pages

PCA-based Multi Task Learning: a Random Matrix Approach

Authors: Malik Tiomoko, Romain Couillet, Frédéric Pascal

Abstract: The article proposes and theoretically analyses a \emph{computationally efficient} multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes \cite{barshan2011supervised,bair2006prediction}. The analysis reveals that (i) by default learning may dramatically fail by suffering from \emph{negative transfer}, but that (ii) simple counter-measure… ▽ More The article proposes and theoretically analyses a \emph{computationally efficient} multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes \cite{barshan2011supervised,bair2006prediction}. The analysis reveals that (i) by default learning may dramatically fail by suffering from \emph{negative transfer}, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks show that the proposed method achieves comparable performance with state-of-the-art MTL methods but at a \emph{significantly reduced computational cost}. △ Less

Submitted 1 November, 2021; originally announced November 2021.

Riemannian classification of EEG signals with missing values

Authors: Alexandre Hippert-Ferrer, Ammar Mian, Florent Bouchard, Frédéric Pascal

Abstract: This paper proposes a strategy to handle missing data for the classification of electroencephalograms using covariance matrices. It relies on the observed-data likelihood within an expectation-maximization algorithm. This approach is compared to two existing state-of-the-art methods: (i) covariance matrices computed with imputed data; (ii) Riemannian averages of partially observed covariance matri… ▽ More This paper proposes a strategy to handle missing data for the classification of electroencephalograms using covariance matrices. It relies on the observed-data likelihood within an expectation-maximization algorithm. This approach is compared to two existing state-of-the-art methods: (i) covariance matrices computed with imputed data; (ii) Riemannian averages of partially observed covariance matrix. All approaches are combined with the minimum distance to Riemannian mean classifier and applied to a classification task of two widely known paradigms of brain-computer interfaces. In addition to be applicable for a wider range of missing data scenarios, the proposed strategy generally performs better than other methods on the considered real EEG data. △ Less

Submitted 5 May, 2022; v1 submitted 19 October, 2021; originally announced October 2021.

Joint Estimation of Location and Scatter in Complex Elliptical Distributions: A robust semiparametric and computationally efficient $R$-estimator of the shape matrix

Authors: Stefano Fortunati, Alexandre Renaux, Frédéric Pascal

Abstract: The joint estimation of the location vector and the shape matrix of a set of independent and identically Complex Elliptically Symmetric (CES) distributed observations is investigated from both the theoretical and computational viewpoints. This joint estimation problem is framed in the original context of semiparametric models allowing us to handle the (generally unknown) density generator as an \t… ▽ More The joint estimation of the location vector and the shape matrix of a set of independent and identically Complex Elliptically Symmetric (CES) distributed observations is investigated from both the theoretical and computational viewpoints. This joint estimation problem is framed in the original context of semiparametric models allowing us to handle the (generally unknown) density generator as an \textit{infinite-dimensional} nuisance parameter. In the first part of the paper, a computationally efficient and memory saving implementation of the robust and semiparmaetric efficient $R$-estimator for shape matrices is derived. Building upon this result, in the second part, a joint estimator, relying on the Tyler's $M$-estimator of location and on the $R$-estimator of shape matrix, is proposed and its Mean Squared Error (MSE) performance compared with the Semiparametric Cramér-Rao Bound (CSCRB). △ Less

Submitted 26 January, 2021; originally announced January 2021.

Comments: This paper has been submitted to the Special Issue (related to MLSP) of the Journal of Signal Processing Systems

Shrinking the eigenvalues of M-estimators of covariance matrix

Authors: Esa Ollila, Daniel P. Palomar, Frédéric Pascal

Abstract: A highly popular regularized (shrinkage) covariance matrix estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward the grand mean of the eigenvalues of the SCM. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adap… ▽ More A highly popular regularized (shrinkage) covariance matrix estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward the grand mean of the eigenvalues of the SCM. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage parameter with minimum mean squared error is proposed. Our approach permits the use of any weight function such as Gaussian, Huber's, Tyler's, or t-weight functions, all of which are commonly used in M-estimation framework. Our simulation examples illustrate that shrinkage M-estimators based on the proposed optimal tuning combined with robust weight function do not loose in performance to shrinkage SCM estimator when the data is Gaussian, but provide significantly improved performance when the data is sampled from an unspecified heavy-tailed elliptically symmetric distribution. Also, real-world and synthetic stock market data validate the performance of the proposed method in practical applications. △ Less

Submitted 28 October, 2020; v1 submitted 17 June, 2020; originally announced June 2020.

Comments: A supplementary report is available at: http://users.spa.aalto.fi/esollila/shrinkM/supplement.pdf

M-estimators of scatter with eigenvalue shrinkage

Authors: Esa Ollila, Daniel P. Palomar, Frederic Pascal

Abstract: A popular regularized (shrinkage) covariance estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward its grand mean. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage… ▽ More A popular regularized (shrinkage) covariance estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward its grand mean. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter matrix and a fully automatic data adaptive method to compute the optimal shrinkage parameter with minimum mean squared error is proposed. Our approach permits the use of any weight function such as Gaussian, Huber's, or $t$ weight functions, all of which are commonly used in M-estimation framework. Our simulation examples illustrate that shrinkage M-estimators based on the proposed optimal tuning combined with robust weight function do not loose in performance to shrinkage SCM estimator when the data is Gaussian, but provide significantly improved performance when the data is sampled from a heavy-tailed distribution. △ Less

Submitted 12 February, 2020; originally announced February 2020.

Comments: To appear in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2020), May 4 - 8, Barcelona, Spain, 2020

A flexible EM-like clustering algorithm for noisy data

Authors: Violeta Roizman, Matthieu Jonckheere, Frédéric Pascal

Abstract: Though very popular, it is well known that the EM for GMM algorithm suffers from non-Gaussian distribution shapes, outliers and high-dimensionality. In this paper, we design a new robust clustering algorithm that can efficiently deal with noise and outliers in diverse data sets. As an EM-like algorithm, it is based on both estimations of clusters centers and covariances. In addition, using a semi-… ▽ More Though very popular, it is well known that the EM for GMM algorithm suffers from non-Gaussian distribution shapes, outliers and high-dimensionality. In this paper, we design a new robust clustering algorithm that can efficiently deal with noise and outliers in diverse data sets. As an EM-like algorithm, it is based on both estimations of clusters centers and covariances. In addition, using a semi-parametric paradigm, the method estimates an unknown scale parameter per data-point. This allows the algorithm to accommodate for heavier tails distributions and outliers without significantly loosing efficiency in various classical scenarios. We first derive and analyze the proposed algorithm in the context of elliptical distributions, showing in particular important insensitivity properties to the underlying data distributions. We then study the convergence and accuracy of the algorithm by considering first synthetic data. Then, we show that the proposed algorithm outperforms other classical unsupervised methods of the literature such as k-means, the EM for Gaussian mixture models and its recent modifications or spectral clustering when applied to real data sets as MNIST, NORB, and 20newsgroups. △ Less

Submitted 5 October, 2020; v1 submitted 2 July, 2019; originally announced July 2019.

On the asymptotics of Maronna's robust PCA

Authors: Gordana Draskovic, Arnaud Breloy, Frederic Pascal

Abstract: The eigenvalue decomposition (EVD) parameters of the second order statistics are ubiquitous in statistical analysis and signal processing. Notably, the EVD of robust scatter $M$-estimators is a popular choice to perform robust probabilistic PCA or other dimension reduction related applications. Towards the goal of characterizing the behavior of these quantities, this paper proposes new asymptotics… ▽ More The eigenvalue decomposition (EVD) parameters of the second order statistics are ubiquitous in statistical analysis and signal processing. Notably, the EVD of robust scatter $M$-estimators is a popular choice to perform robust probabilistic PCA or other dimension reduction related applications. Towards the goal of characterizing the behavior of these quantities, this paper proposes new asymptotics for the EVD parameters (i.e. eigenvalues, eigenvectors and principal subspace) of the scatter $M$-estimator in the context of complex elliptically symmetric distributions. First, their Gaussian asymptotic distribution is obtained by extending standard results on the sample covariance matrix in a Gaussian context. Second, their convergence rate towards the EVD parameters of a Gaussian-Core Wishart Equivalent is derived. This second result represents the main contribution in the sense that it quantifies when it is acceptable to directly plug-in well-established results on the EVD of Wishart-distributed matrix for characterizing the EVD of $M$-estimators. Eventually, some examples (low-rank adaptive filtering and Intrinsic bias analysis) are provided to illustrate where the obtained results can be leveraged. △ Less

Submitted 5 November, 2018; originally announced November 2018.

Improving Portfolios Global Performance with Robust Covariance Matrix Estimation: Application to the Maximum Variety Portfolio

Authors: Emmanuelle Jay, Eugénie Terreaux, Jean-Philippe Ovarlez, Frédéric Pascal

Abstract: This paper presents how the most recent improvements made on covariance matrix estimation and model order selection can be applied to the portfolio optimisation problem. The particular case of the Maximum Variety Portfolio is treated but the same improvements apply also in the other optimisation problems such as the Minimum Variance Portfolio. We assume that the most important information (or the… ▽ More This paper presents how the most recent improvements made on covariance matrix estimation and model order selection can be applied to the portfolio optimisation problem. The particular case of the Maximum Variety Portfolio is treated but the same improvements apply also in the other optimisation problems such as the Minimum Variance Portfolio. We assume that the most important information (or the latent factors) are embedded in correlated Elliptical Symmetric noise extending classical Gaussian assumptions. We propose here to focus on a recent method of model order selection allowing to efficiently estimate the subspace of main factors describing the market. This non-standard model order selection problem is solved through Random Matrix Theory and robust covariance matrix estimation. The proposed procedure will be explained through synthetic data and be applied and compared with standard techniques on real market data showing promising improvements. △ Less

Submitted 31 March, 2018; originally announced April 2018.

Comments: Submitted to 26th European Signal Processing Conference (EUSIPCO 2018)

New insights into the statistical properties of $M$-estimators

Authors: Gordana Draskovic, Frederic Pascal

Abstract: This paper proposes an original approach to better understanding the behavior of robust scatter matrix $M$-estimators. Scatter matrices are of particular interest for many signal processing applications since the resulting performance strongly relies on the quality of the matrix estimation. In this context, $M$-estimators appear as very interesting candidates, mainly due to their flexibility to th… ▽ More This paper proposes an original approach to better understanding the behavior of robust scatter matrix $M$-estimators. Scatter matrices are of particular interest for many signal processing applications since the resulting performance strongly relies on the quality of the matrix estimation. In this context, $M$-estimators appear as very interesting candidates, mainly due to their flexibility to the statistical model and their robustness to outliers and/or missing data. However, the behavior of such estimators still remains unclear and not well understood since they are described by fixed-point equations that make their statistical analysis very difficult. To fill this gap, the main contribution of this work is to prove that these estimators distribution is more accurately described by a Wishart distribution than by the classical asymptotical Gaussian approximation. To that end, we propose a new `Gaussian-core' representation for Complex Elliptically Symmetric (CES) distributions and we analyze the proximity between $M$-estimators and a Gaussian-based Sample Covariance Matrix (SCM), unobservable in practice and playing only a theoretical role. To confirm our claims we also provide results for a widely used function of $M$-estimators, the Mahalanobis distance. Finally, Monte Carlo simulations for various scenarios are presented to validate theoretical results. △ Less

Submitted 5 November, 2018; v1 submitted 26 October, 2017; originally announced October 2017.

Journal ref: IEEE Transactions on Signal Processing, vol. 66, pp. 4253-4263, August 2018

Robust Model Order Selection in Large Dimensional Elliptically Symmetric Noise

Authors: Eugénie Terreaux, Jean-Philippe Ovarlez, Frédéric Pascal

Abstract: This paper deals with model order selection in context of correlated noise. More precisely, one considers sources embedded in an additive Complex Elliptically Symmetric (CES) noise, with unknown parameters. The main difficultly for estimating the model order lies into the noise correlation, namely the scatter matrix of the corresponding CES distribution. In this work, to tackle that problem, one a… ▽ More This paper deals with model order selection in context of correlated noise. More precisely, one considers sources embedded in an additive Complex Elliptically Symmetric (CES) noise, with unknown parameters. The main difficultly for estimating the model order lies into the noise correlation, namely the scatter matrix of the corresponding CES distribution. In this work, to tackle that problem, one adopts a two-step approach: first, we develop two different methods based on a Toeplitz-structured model for estimating this unknown scatter matrix and for whitening the correlated noise. Then, we apply Maronna's $M$-estimators on the whitened signal to estimate the covariance matrix of the "decorrelated" signal in order to estimate the model order. The proposed methodology is based both on robust estimation theory as well as large Random Matrix Theory, and original results are derived, proving the efficiency of this methodology. Indeed, the main theoretical contribution is to derive consistent robust estimators for the covariance matrix of the signal-plus-correlated noise in a large dimensional regime and to propose efficient methodology to estimate the rank of signal subspace. Finally, as shown in the analysis, these results show a great improvement compared to the state-of-the-art, on both simulated and real hyperspectral images. △ Less

Submitted 18 October, 2017; originally announced October 2017.

Convergence of Structured Quadratic Forms With Application to Theoretical Performances of Adaptive Filters in Low Rank Gaussian Context

Authors: Alice Combernoux, Frederic Pascal, Guillaume Ginolhac, Marc Lesturgie

Abstract: This paper addresses the problem of deriving the asymptotic performance of adaptive Low Rank (LR) filters used in target detection embedded in a disturbance composed of a LR Gaussian noise plus a white Gaussian noise. In this context, we use the Signal to Interference to Noise Ratio (SINR) loss as performance measure which is a function of the estimated projector onto the LR noise subspace. Howeve… ▽ More This paper addresses the problem of deriving the asymptotic performance of adaptive Low Rank (LR) filters used in target detection embedded in a disturbance composed of a LR Gaussian noise plus a white Gaussian noise. In this context, we use the Signal to Interference to Noise Ratio (SINR) loss as performance measure which is a function of the estimated projector onto the LR noise subspace. However, although the SINR loss can be determined through Monte-Carlo simulations or real data, this process remains quite time consuming. Thus, this paper proposes to predict the SINR loss behavior in order to not depend on the data anymore and be quicker. To derive this theoretical result, previous works used a restrictive hypothesis assuming that the target is orthogonal to the LR noise. In this paper, we propose to derive this theoretical performance by relaxing this hypothesis and using Random Matrix Theory (RMT) tools. These tools will be used to present the convergences of simple quadratic forms and perform new RMT convergences of structured quadratic forms and SINR loss in the large dimensional regime, i.e. the size and the number of the data tend to infinity at the same rate. We show through simulations the interest of our approach compared to the previous works when the restrictive hypothesis is no longer verified. △ Less

Submitted 18 March, 2015; originally announced March 2015.

Adaptive non-Zero Mean Gaussian Detection and Application to Hyperspectral Imaging

Authors: Joana Frontera-Pons, Frederic Pascal, Jean-Philippe Ovarlez

Abstract: Classical target detection schemes are usually obtained deriving the likelihood ratio under Gaussian hypothesis and replacing the unknown background parameters by their estimates. In most applications, interference signals are assumed to be Gaussian with zero mean or with a known mean vector that can be removed and with unknown covariance matrix. When mean vector is unknown, it has to be jointly e… ▽ More Classical target detection schemes are usually obtained deriving the likelihood ratio under Gaussian hypothesis and replacing the unknown background parameters by their estimates. In most applications, interference signals are assumed to be Gaussian with zero mean or with a known mean vector that can be removed and with unknown covariance matrix. When mean vector is unknown, it has to be jointly estimated with the covariance matrix, as it is the case for instance in hyperspectral imaging. In this paper, the adaptive versions of the classical Matched Filter and the Normalized Matched Filter, as well as two versions of the Kelly detector are first derived and then are analyzed for the case when the mean vector of the background is unknown. More precisely, theoretical closed-form expressions for false-alarm regulation are derived and the Constant False Alarm Rate property is pursued to allow the detector to be independent of nuisance parameters. Finally, the theoretical contribution is validated through simulations and on real hyperspectral scenes. △ Less

Submitted 10 April, 2014; originally announced April 2014.

On the convergence of Maronna's $M$-estimators of scatter

Authors: Yacine Chitour, Romain Couillet, Frederic Pascal

Abstract: In this paper, {we propose an alternative proof for the uniqueness} of Maronna's $M$-estimator of scatter (Maronna, 1976) for $N$ vector observations $\mathbf y_1,...,\mathbf y_N\in\mathbb R^m$ under a mild constraint of linear independence of any subset of $m$ of these vectors. This entails in particular almost sure uniqueness for random vectors $\mathbf y_i$ with a density as long as $N>m$. {Thi… ▽ More In this paper, {we propose an alternative proof for the uniqueness} of Maronna's $M$-estimator of scatter (Maronna, 1976) for $N$ vector observations $\mathbf y_1,...,\mathbf y_N\in\mathbb R^m$ under a mild constraint of linear independence of any subset of $m$ of these vectors. This entails in particular almost sure uniqueness for random vectors $\mathbf y_i$ with a density as long as $N>m$. {This approach allows to establish further relations that demonstrate that a properly normalized Tyler's $M$-estimator of scatter (Tyler, 1987) can be considered as a limit of Maronna's $M$-estimator. More precisely, the contribution is to show that each $M$-estimator converges towards a particular Tyler's $M$-estimator.} These results find important implications in recent works on the large dimensional (random matrix) regime of robust $M$-estimation. △ Less

Submitted 4 November, 2014; v1 submitted 24 March, 2014; originally announced March 2014.

Generalized robust shrinkage estimator and its application to STAP detection problem

Authors: Frederic Pascal, Yacine Chitour, Yihui Quek

Abstract: Recently, in the context of covariance matrix estimation, in order to improve as well as to regularize the performance of the Tyler's estimator [1] also called the Fixed-Point Estimator (FPE) [2], a "shrinkage" fixed-point estimator has been introduced in [3]. First, this work extends the results of [3,4] by giving the general solution of the "shrinkage" fixed-point algorithm. Secondly, by analyzi… ▽ More Recently, in the context of covariance matrix estimation, in order to improve as well as to regularize the performance of the Tyler's estimator [1] also called the Fixed-Point Estimator (FPE) [2], a "shrinkage" fixed-point estimator has been introduced in [3]. First, this work extends the results of [3,4] by giving the general solution of the "shrinkage" fixed-point algorithm. Secondly, by analyzing this solution, called the generalized robust shrinkage estimator, we prove that this solution converges to a unique solution when the shrinkage parameter $β$ (losing factor) tends to 0. This solution is exactly the FPE with the trace of its inverse equal to the dimension of the problem. This general result allows one to give another interpretation of the FPE and more generally, on the Maximum Likelihood approach for covariance matrix estimation when constraints are added. Then, some simulations illustrate our theoretical results as well as the way to choose an optimal shrinkage factor. Finally, this work is applied to a Space-Time Adaptive Processing (STAP) detection problem on real STAP data. △ Less

Submitted 15 September, 2014; v1 submitted 26 November, 2013; originally announced November 2013.

Parameter Estimation For Multivariate Generalized Gaussian Distributions

Authors: F. Pascal, L. Bombrun, J. Y. Tourneret, Y. Berthoumieu

Abstract: Due to its heavy-tailed and fully parametric form, the multivariate generalized Gaussian distribution (MGGD) has been receiving much attention for modeling extreme events in signal and image processing applications. Considering the estimation issue of the MGGD parameters, the main contribution of this paper is to prove that the maximum likelihood estimator (MLE) of the scatter matrix exists and is… ▽ More Due to its heavy-tailed and fully parametric form, the multivariate generalized Gaussian distribution (MGGD) has been receiving much attention for modeling extreme events in signal and image processing applications. Considering the estimation issue of the MGGD parameters, the main contribution of this paper is to prove that the maximum likelihood estimator (MLE) of the scatter matrix exists and is unique up to a scalar factor, for a given shape parameter β\in(0,1). Moreover, an estimation algorithm based on a Newton-Raphson recursion is proposed for computing the MLE of MGGD parameters. Various experiments conducted on synthetic and real data are presented to illustrate the theoretical derivations in terms of number of iterations and number of samples for different values of the shape parameter. The main conclusion of this work is that the parameters of MGGDs can be estimated using the maximum likelihood principle with good performance. △ Less

Submitted 24 February, 2017; v1 submitted 26 February, 2013; originally announced February 2013.

Asymptotic properties of robust complex covariance matrix estimates

Authors: Melanie Mahot, Philippe Forster, Frederic Pascal, Jean-Philippe Ovarlez

Abstract: In many statistical signal processing applications, the estimation of nuisance parameters and parameters of interest is strongly linked to the resulting performance. Generally, these applications deal with complex data. This paper focuses on covariance matrix estimation problems in non-Gaussian environments and particularly, the M-estimators in the context of elliptical distributions. Firstly, thi… ▽ More In many statistical signal processing applications, the estimation of nuisance parameters and parameters of interest is strongly linked to the resulting performance. Generally, these applications deal with complex data. This paper focuses on covariance matrix estimation problems in non-Gaussian environments and particularly, the M-estimators in the context of elliptical distributions. Firstly, this paper extends to the complex case the results of Tyler in [1]. More precisely, the asymptotic distribution of these estimators as well as the asymptotic distribution of any homogeneous function of degree 0 of the M-estimates are derived. On the other hand, we show the improvement of such results on two applications: DOA (directions of arrival) estimation using the MUSIC (MUltiple SIgnal Classification) algorithm and adaptive radar detection based on the ANMF (Adaptive Normalized Matched Filter) test. △ Less

Submitted 6 November, 2012; v1 submitted 5 September, 2012; originally announced September 2012.