Showing 1–19 of 19 results for author: Borsoi, R

A Generalized Multiscale Bundle-Based Hyperspectral Sparse Unmixing Algorithm

Authors: Luciano Carvalho Ayres, Ricardo Augusto Borsoi, José Carlos Moreira Bermudez, Sérgio José Melo de Almeida

Abstract: In hyperspectral sparse unmixing, a successful approach employs spectral bundles to address the variability of the endmembers in the spatial domain. However, the regularization penalties usually employed aggregate substantial computational complexity, and the solutions are very noise-sensitive. We generalize a multiscale spatial regularization approach to solve the unmixing problem by incorporatin… ▽ More In hyperspectral sparse unmixing, a successful approach employs spectral bundles to address the variability of the endmembers in the spatial domain. However, the regularization penalties usually employed aggregate substantial computational complexity, and the solutions are very noise-sensitive. We generalize a multiscale spatial regularization approach to solve the unmixing problem by incorporating group sparsity-inducing mixed norms. Then, we propose a noise-robust method that can take advantage of the bundle structure to deal with endmember variability while ensuring inter- and intra-class sparsity in abundance estimation with reasonable computational cost. We also present a general heuristic to select the \emph{most representative} abundance estimation over multiple runs of the unmixing process, yielding a solution that is robust and highly reproducible. Experiments illustrate the robustness and consistency of the results when compared to related methods. △ Less

Submitted 23 January, 2024; originally announced January 2024.

On the Impact of Sampling on Deep Sequential State Estimation

Authors: Helena Calatrava, Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas

Abstract: State inference and parameter learning in sequential models can be successfully performed with approximation techniques that maximize the evidence lower bound to the marginal log-likelihood of the data distribution. These methods may be referred to as Dynamical Variational Autoencoders, and our specific focus lies on the deep Kalman filter. It has been shown that the ELBO objective can oversimplif… ▽ More State inference and parameter learning in sequential models can be successfully performed with approximation techniques that maximize the evidence lower bound to the marginal log-likelihood of the data distribution. These methods may be referred to as Dynamical Variational Autoencoders, and our specific focus lies on the deep Kalman filter. It has been shown that the ELBO objective can oversimplify data representations, potentially compromising estimation quality. Tighter Monte Carlo objectives have been proposed in the literature to enhance generative modeling performance. For instance, the IWAE objective uses importance weights to reduce the variance of marginal log-likelihood estimates. In this paper, importance sampling is applied to the DKF framework for learning deep Markov models, resulting in the IW-DKF, which shows an improvement in terms of log-likelihood estimates and KL divergence between the variational distribution and the transition model. The framework using the sampled DKF update rule is also accommodated to address sequential state and parameter estimation when working with highly non-linear physics-based models. An experiment with the 3-space Lorenz attractor shows an enhanced generative modeling performance and also a decrease in RMSE when estimating the model parameters and latent states, indicating that tighter MCOs lead to improved state inference performance. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Comments: To appear in the Proceedings of the Asilomar Conference on Signals, Systems, and Computers, October 2023, 5 pages, 3 figures, 3 tables

Learning Interpretable Deep Disentangled Neural Networks for Hyperspectral Unmixing

Authors: Ricardo Augusto Borsoi, Deniz Erdoğmuş, Tales Imbiriba

Abstract: Although considerable effort has been dedicated to improving the solution to the hyperspectral unmixing problem, non-idealities such as complex radiation scattering and endmember variability negatively impact the performance of most existing algorithms and can be very challenging to address. Recently, deep learning-based frameworks have been explored for hyperspectral umixing due to their flexibil… ▽ More Although considerable effort has been dedicated to improving the solution to the hyperspectral unmixing problem, non-idealities such as complex radiation scattering and endmember variability negatively impact the performance of most existing algorithms and can be very challenging to address. Recently, deep learning-based frameworks have been explored for hyperspectral umixing due to their flexibility and powerful representation capabilities. However, such techniques either do not address the non-idealities of the unmixing problem, or rely on black-box models which are not interpretable. In this paper, we propose a new interpretable deep learning method for hyperspectral unmixing that accounts for nonlinearity and endmember variability. The proposed method leverages a probabilistic variational deep-learning framework, where disentanglement learning is employed to properly separate the abundances and endmembers. The model is learned end-to-end using stochastic backpropagation, and trained using a self-supervised strategy which leverages benefits from semi-supervised learning techniques. Furthermore, the model is carefully designed to provide a high degree of interpretability. This includes modeling the abundances as a Dirichlet distribution, the endmembers using low-dimensional deep latent variable representations, and using two-stream neural networks composed of additive piecewise-linear/nonlinear components. Experimental results on synthetic and real datasets illustrate the performance of the proposed method compared to state-of-the-art algorithms. △ Less

Submitted 3 October, 2023; originally announced October 2023.

Dynamical Hyperspectral Unmixing with Variational Recurrent Neural Networks

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas

Abstract: Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. It reveals the dynamical evolution of the materials (endmembers) and of their proportions (abundances) in a given scene. However, adequately accounting for the spatial and temporal variability of the endmembers in MTHU is challenging, and has not been fully addressed so far in unsupe… ▽ More Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. It reveals the dynamical evolution of the materials (endmembers) and of their proportions (abundances) in a given scene. However, adequately accounting for the spatial and temporal variability of the endmembers in MTHU is challenging, and has not been fully addressed so far in unsupervised frameworks. In this work, we propose an unsupervised MTHU algorithm based on variational recurrent neural networks. First, a stochastic model is proposed to represent both the dynamical evolution of the endmembers and their abundances, as well as the mixing process. Moreover, a new model based on a low-dimensional parametrization is used to represent spatial and temporal endmember variability, significantly reducing the amount of variables to be estimated. We propose to formulate MTHU as a Bayesian inference problem. However, the solution to this problem does not have an analytical solution due to the nonlinearity and non-Gaussianity of the model. Thus, we propose a solution based on deep variational inference, in which the posterior distribution of the estimated abundances and endmembers is represented by using a combination of recurrent neural networks and a physically motivated model. The parameters of the model are learned using stochastic backpropagation. Experimental results show that the proposed method outperforms state of the art MTHU algorithms. △ Less

Submitted 19 March, 2023; originally announced March 2023.

Online Fusion of Multi-resolution Multispectral Images with Weakly Supervised Temporal Dynamics

Authors: Haoqing Li, Bhavya Duvvuri, Ricardo Borsoi, Tales Imbiriba, Edward Beighley, Deniz Erdogmus, Pau Closas

Abstract: Real-time satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena such as floods, earthquakes, etc. One important constraint of satellite imaging is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In… ▽ More Real-time satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena such as floods, earthquakes, etc. One important constraint of satellite imaging is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In this paper, we focus on fusing multi-temporal, multi-spectral images where data acquired from different instruments with different spatial resolutions is used. We leverage the spatial relationship between images at multiple modalities to generate high-resolution image sequences at higher revisiting rates. To achieve this goal, we formulate the fusion method as a recursive state estimation problem and study its performance in filtering and smoothing contexts. Furthermore, a calibration strategy is proposed to estimate the time-varying temporal dynamics of the image sequence using only a small amount of historical image data. Differently from the training process in traditional machine learning algorithms, which usually require large datasets and computation times, the parameters of the temporal dynamical model are calibrated based on an analytical expression that uses only two of the images in the historical dataset. A distributed version of the Bayesian filtering and smoothing strategies is also proposed to reduce its computational complexity. To evaluate the proposed methodology we consider a water map** task where real data acquired by the Landsat and MODIS instruments are fused generating high spatial-temporal resolution image estimates. Our experiments show that the proposed methodology outperforms the competing methods in both estimation accuracy and water map** tasks. △ Less

Submitted 6 January, 2023; originally announced January 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2204.12566

Recursive classification of satellite imaging time-series: An application to land cover map**

Authors: Helena Calatrava, Bhavya Duvvuri, Haoqing Li, Ricardo Borsoi, Edward Beighley, Deniz Erdogmus, Pau Closas, Tales Imbiriba

Abstract: A wide variety of applications of fundamental importance for security, environmental protection and urban development need access to accurate land cover monitoring, for which the analysis of optical remote sensing imagery is key. Classification of time-series images, particularly with recursive methods, is of increasing interest in the current literature. Nevertheless, existing recursive approache… ▽ More A wide variety of applications of fundamental importance for security, environmental protection and urban development need access to accurate land cover monitoring, for which the analysis of optical remote sensing imagery is key. Classification of time-series images, particularly with recursive methods, is of increasing interest in the current literature. Nevertheless, existing recursive approaches typically require large amounts of training data. This paper introduces a recursive classification framework that improves the decision-making process in multitemporal and multispectral land cover classification algorithms while requiring low computational cost and minimal supervision. The proposed approach allows the conversion of an instantaneous classifier into a recursive Bayesian classifier by using a probabilistic framework that is robust to non-informative image variations. Three two-class experiments are conducted using Sentinel-2 data. The first one consists in the water map** of an embankment dam in Oroville, California, USA; the second one is a water map** experiment of the Charles river basin area in Boston, Massachusetts, USA; and the last experiment addresses deforestation detection in the Amazon rainforest. The spectral index classifier (SIC) is introduced as a method to convert broadband spectral indices such as the NDWI and the MNDWI into probabilistic classification results. A classifier based on the Gaussian mixture model (GMM), a logistic regression (LR) classifier, and the SIC are compared to their recursive counterparts. Results show that the use of Bayesian recursion significantly increases the robustness of existing instantaneous classifiers in multitemporal settings, including two state-of-the-art deep learning-based models, without the need for additional training data. Furthermore, the proposed framework scales well for an increasing number of classes. △ Less

Submitted 7 January, 2024; v1 submitted 4 January, 2023; originally announced January 2023.

Comments: Without supplemental results: 30 pages, 17 figures, 4 tables. With supplemental results: 39 pages, 26 figures, 4 tables

Coupled CP tensor decomposition with shared and distinct components for multi-task fMRI data fusion

Authors: Ricardo Augusto Borsoi, Isabell Lehmann, Mohammad Abu Baker Siddique Akhonda, Vince Calhoun, Konstantin Usevich, David Brie, Tülay Adali

Abstract: Discovering components that are shared in multiple datasets, next to dataset-specific features, has great potential for studying the relationships between different subjects or tasks in functional Magnetic Resonance Imaging (fMRI) data. Coupled matrix and tensor factorization approaches have been useful for flexible data fusion, or decomposition to extract features that can be used in multiple way… ▽ More Discovering components that are shared in multiple datasets, next to dataset-specific features, has great potential for studying the relationships between different subjects or tasks in functional Magnetic Resonance Imaging (fMRI) data. Coupled matrix and tensor factorization approaches have been useful for flexible data fusion, or decomposition to extract features that can be used in multiple ways. However, existing methods do not directly recover shared and dataset-specific components, which requires post-processing steps involving additional hyperparameter selection. In this paper, we propose a tensor-based framework for multi-task fMRI data fusion, using a partially constrained canonical polyadic (CP) decomposition model. Differently from previous approaches, the proposed method directly recovers shared and dataset-specific components, leading to results that are directly interpretable. A strategy to select a highly reproducible solution to the decomposition is also proposed. We evaluate the proposed methodology on real fMRI data of three tasks, and show that the proposed method finds meaningful components that clearly identify group differences between patients with schizophrenia and healthy controls. △ Less

Submitted 23 July, 2023; v1 submitted 25 November, 2022; originally announced November 2022.

Deep Hyperspectral and Multispectral Image Fusion with Inter-image Variability

Authors: Xiuheng Wang, Ricardo Augusto Borsoi, Cédric Richard, Jie Chen

Abstract: Hyperspectral and multispectral image fusion allows us to overcome the hardware limitations of hyperspectral imaging systems inherent to their lower spatial resolution. Nevertheless, existing algorithms usually fail to consider realistic image acquisition conditions. This paper presents a general imaging model that considers inter-image variability of data from heterogeneous sources and flexible i… ▽ More Hyperspectral and multispectral image fusion allows us to overcome the hardware limitations of hyperspectral imaging systems inherent to their lower spatial resolution. Nevertheless, existing algorithms usually fail to consider realistic image acquisition conditions. This paper presents a general imaging model that considers inter-image variability of data from heterogeneous sources and flexible image priors. The fusion problem is stated as an optimization problem in the maximum a posteriori framework. We introduce an original image fusion method that, on the one hand, solves the optimization problem accounting for inter-image variability with an iteratively reweighted scheme and, on the other hand, that leverages light-weight CNN-based networks to learn realistic image priors from data. In addition, we propose a zero-shot strategy to directly learn the image-specific prior of the latent images in an unsupervised manner. The performance of the algorithm is illustrated with real data subject to inter-image variability. △ Less

Submitted 11 April, 2023; v1 submitted 24 August, 2022; originally announced August 2022.

Comments: IEEE Trans. Geosci. Remote sens., to be published. Manuscript submitted August 23, 2022; revised Dec. 15, 2022, and Mar. 13, 2023; and accepted Apr. 07, 2023

Online multi-resolution fusion of space-borne multispectral images

Authors: Haoqing Li, Bhavia Duvviri, Ricardo Borsoi, Tales Imbiriba, Edward Beighley, Deniz Erdogmus, Pau Closas

Abstract: Satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena. One important feature of satellite images is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In this paper, we focus on fusing multi-temporal,… ▽ More Satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena. One important feature of satellite images is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In this paper, we focus on fusing multi-temporal, multi-spectral images where data acquired from different instruments with different spatial resolutions is used. We leverage the spatial relationship between images at multiple modalities to generate high-resolution image sequences at higher revisiting rates. To achieve this goal, we formulate the fusion method as a recursive state estimation problem and study its performance in filtering and smoothing contexts. The proposed strategy clearly outperforms competing methodologies, which is shown in the paper for real data acquired by the Landsat and MODIS instruments. △ Less

Submitted 26 April, 2022; originally announced April 2022.

Comments: Published in IEEE AeroConf 2022

Graph topology inference with derivative-reproducing property in RKHS: algorithm and convergence analysis

Authors: Mircea Moscu, Ricardo A. Borsoi, Cédric Richard, José-Carlos M. Bermudez

Abstract: In many areas such as computational biology, finance or social sciences, knowledge of an underlying graph explaining the interactions between agents is of paramount importance but still challenging. Considering that these interactions may be based on nonlinear relationships adds further complexity to the topology inference problem. Among the latest methods that respond to this need is a topology i… ▽ More In many areas such as computational biology, finance or social sciences, knowledge of an underlying graph explaining the interactions between agents is of paramount importance but still challenging. Considering that these interactions may be based on nonlinear relationships adds further complexity to the topology inference problem. Among the latest methods that respond to this need is a topology inference one proposed by the authors, which estimates a possibly directed adjacency matrix in an online manner. Contrasting with previous approaches based on linear models, the considered model is able to explain nonlinear interactions between the agents in a network. The novelty in the considered method is the use of a derivative-reproducing property to enforce network sparsity, while reproducing kernels are used to model the nonlinear interactions. The aim of this paper is to present a thorough convergence analysis of this method. The analysis is proven to be sane both in the mean and mean square sense. In addition, stability conditions are devised to ensure the convergence of the analyzed method. △ Less

Submitted 28 April, 2021; originally announced April 2021.

Comments: 24 pages, 11 of which of which are supplementary material; 4 figures

Robust parameter design for Wiener-based binaural noise reduction methods in hearing aids

Authors: Diego M. Carmo, Ricardo Borsoi, Márcio H. Costa

Abstract: This work presents a method for designing the weighting parameter required by Wiener-based binaural noise reduction methods. This parameter establishes the desired tradeoff between noise reduction and binaural cue preservation in hearing aid applications. The proposed strategy was specially derived for the preservation of interaural level difference, interaural time difference and interaural coher… ▽ More This work presents a method for designing the weighting parameter required by Wiener-based binaural noise reduction methods. This parameter establishes the desired tradeoff between noise reduction and binaural cue preservation in hearing aid applications. The proposed strategy was specially derived for the preservation of interaural level difference, interaural time difference and interaural coherence binaural cues. It is defined as a function of the average input noise power at the microphones, providing robustness against the influence of joint changes in noise and speech power (Lombard effect), as well as to signal to noise ratio (SNR) variations. A theoretical framework, based on the mathematical definition of the homogeneity degree, is presented and applied to a generic augmented Wiener-based cost function. The theoretical insights obtained are supported bycomputational simulations and psychoacoustic experiments using the multichannel Wiener filter with interaural transfer function preservation technique (MWF-ITF), as a case study. Statistical analysis indicates that the proposed dynamic structure for the weighting parameter and the design method of its fixed part provide significant robustness against changes in the original binaural cues of both speech and residual noise, at the cost of a small decrease in the noise reduction performance, as compared to the use of a purely fixed weighting parameter. △ Less

Submitted 19 April, 2021; originally announced April 2021.

Model-Based Deep Autoencoder Networks for Nonlinear Hyperspectral Unmixing

Authors: Haoqing Li, Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas, José Carlos Moreira Bermudez, Deniz Erdoğmuş

Abstract: Autoencoder (AEC) networks have recently emerged as a promising approach to perform unsupervised hyperspectral unmixing (HU) by associating the latent representations with the abundances, the decoder with the mixing model and the encoder with its inverse. AECs are especially appealing for nonlinear HU since they lead to unsupervised and model-free algorithms. However, existing approaches fail to e… ▽ More Autoencoder (AEC) networks have recently emerged as a promising approach to perform unsupervised hyperspectral unmixing (HU) by associating the latent representations with the abundances, the decoder with the mixing model and the encoder with its inverse. AECs are especially appealing for nonlinear HU since they lead to unsupervised and model-free algorithms. However, existing approaches fail to explore the fact that the encoder should invert the mixing process, which might reduce their robustness. In this paper, we propose a model-based AEC for nonlinear HU by considering the mixing model a nonlinear fluctuation over a linear mixture. Differently from previous works, we show that this restriction naturally imposes a particular structure to both the encoder and to the decoder networks. This introduces prior information in the AEC without reducing the flexibility of the mixing model. Simulations with synthetic and real data indicate that the proposed strategy improves nonlinear HU. △ Less

Submitted 16 April, 2021; originally announced April 2021.

Fast Unmixing and Change Detection in Multitemporal Hyperspectral Data

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard

Abstract: Multitemporal spectral unmixing (SU) is a powerful tool to process hyperspectral image (HI) sequences due to its ability to reveal the evolution of materials over time and space in a scene. However, significant spectral variability is often observed between collection of images due to variations in acquisition or seasonal conditions. This characteristic has to be considered in the design of SU alg… ▽ More Multitemporal spectral unmixing (SU) is a powerful tool to process hyperspectral image (HI) sequences due to its ability to reveal the evolution of materials over time and space in a scene. However, significant spectral variability is often observed between collection of images due to variations in acquisition or seasonal conditions. This characteristic has to be considered in the design of SU algorithms. Because of its good performance, the multiple endmember spectral mixture analysis algorithm (MESMA) has been recently used to perform SU in multitemporal scenarios arising in several practical applications. However, MESMA does not consider the relationship between the different HIs, and its computational complexity is extremely high for large spectral libraries. In this work, we propose an efficient multitemporal SU method that exploits the high temporal correlation between the abundances to provide more accurate results at a lower computational complexity. We propose to solve the complex general multitemporal SU problem by separately addressing the endmember selection and the abundance estimation problems. This leads to a simpler solution without sacrificing the accuracy of the results. We also propose a strategy to detect and address abrupt abundance variations in time. Theoretical results demonstrate how the proposed method compares to MESMA in terms of quality, and how effective it is in detecting abundance changes. This analysis provides valuable insight into the conditions under which the algorithm succeeds. Simulation results show that the proposed method achieves state-of-the-art performance at a smaller computational cost. △ Less

Submitted 9 September, 2021; v1 submitted 6 April, 2021; originally announced April 2021.

Tech Report: A Homogeneity-Based Multiscale Hyperspectral Image Representation for Sparse Spectral Unmixing

Authors: L. C. Ayres, S. J. M. de Almeida, J. C. M. Bermudez, R. A. Borsoi

Abstract: Several approaches have been proposed to solve the spectral unmixing problem in hyperspectral image analysis. Among them the use of sparse regression techniques aims to characterize the abundances in pixels based on a large library of spectral signatures known a priori. Recently, the integration of image spatial-contextual information significantly enhanced the performance of sparse unmixing. In t… ▽ More Several approaches have been proposed to solve the spectral unmixing problem in hyperspectral image analysis. Among them the use of sparse regression techniques aims to characterize the abundances in pixels based on a large library of spectral signatures known a priori. Recently, the integration of image spatial-contextual information significantly enhanced the performance of sparse unmixing. In this work, we propose a computationally efficient multiscale representation method for hyperspectral data adapted to the unmixing problem. The proposed method is based on a hierarchical extension of the SLIC oversegmentation algorithm constructed using a robust homogeneity testing. The image is subdivided into a set of spectrally homogeneous regions formed by pixels with similar characteristics (superpixels). This representation is then used to provide prior spatial regularity information for the abundances of materials present in the scene, improving the conditioning of the unmixing problem. Simulation results illustrate that the method is capable of estimating abundances with high quality and low computational cost, especially in noisy scenarios. △ Less

Submitted 10 February, 2021; originally announced February 2021.

Coupled Tensor Decomposition for Hyperspectral and Multispectral Image Fusion with Inter-Image Variability

Authors: Ricardo Augusto Borsoi, Clémence Prévost, Konstantin Usevich, David Brie, José Carlos Moreira Bermudez, Cédric Richard

Abstract: Coupled tensor approximation has recently emerged as a promising approach for the fusion of hyperspectral and multispectral images, reconciling state of the art performance with strong theoretical guarantees. However, tensor-based approaches previously proposed assume that the different observed images are acquired under exactly the same conditions. A recent work proposed to accommodate inter-imag… ▽ More Coupled tensor approximation has recently emerged as a promising approach for the fusion of hyperspectral and multispectral images, reconciling state of the art performance with strong theoretical guarantees. However, tensor-based approaches previously proposed assume that the different observed images are acquired under exactly the same conditions. A recent work proposed to accommodate inter-image spectral variability in the image fusion problem using a matrix factorization-based formulation, but did not account for spatially-localized variations. Moreover, it lacks theoretical guarantees and has a high associated computational complexity. In this paper, we consider the image fusion problem while accounting for both spatially and spectrally localized changes in an additive model. We first study how the general identifiability of the model is impacted by the presence of such changes. Then, assuming that the high-resolution image and the variation factors admit a Tucker decomposition, two new algorithms are proposed -- one purely algebraic, and another based on an optimization procedure. Theoretical guarantees for the exact recovery of the high-resolution image are provided for both algorithms. Experimental results show that the proposed method outperforms state-of-the-art methods in the presence of spectral and spatial variations between the images, at a smaller computational cost. △ Less

Submitted 5 December, 2020; v1 submitted 30 June, 2020; originally announced June 2020.

Online Graph-Based Change Point Detection in Multiband Image Sequences

Authors: Ricardo Augusto Borsoi, Cédric Richard, André Ferrari, Jie Chen, José Carlos Moreira Bermudez

Abstract: The automatic detection of changes or anomalies between multispectral and hyperspectral images collected at different time instants is an active and challenging research topic. To effectively perform change-point detection in multitemporal images, it is important to devise techniques that are computationally efficient for processing large datasets, and that do not require knowledge about the natur… ▽ More The automatic detection of changes or anomalies between multispectral and hyperspectral images collected at different time instants is an active and challenging research topic. To effectively perform change-point detection in multitemporal images, it is important to devise techniques that are computationally efficient for processing large datasets, and that do not require knowledge about the nature of the changes. In this paper, we introduce a novel online framework for detecting changes in multitemporal remote sensing images. Acting on neighboring spectra as adjacent vertices in a graph, this algorithm focuses on anomalies concurrently activating groups of vertices corresponding to compact, well-connected and spectrally homogeneous image regions. It fully benefits from recent advances in graph signal processing to exploit the characteristics of the data that lie on irregular supports. Moreover, the graph is estimated directly from the images using superpixel decomposition algorithms. The learning algorithm is scalable in the sense that it is efficient and spatially distributed. Experiments illustrate the detection and localization performance of the method. △ Less

Submitted 24 June, 2020; originally announced June 2020.

Spectral Variability in Hyperspectral Data Unmixing: A Comprehensive Review

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard, Jocelyn Chanussot, Lucas Drumetz, Jean-Yves Tourneret, Alina Zare, Christian Jutten

Abstract: The spectral signatures of the materials contained in hyperspectral images, also called endmembers (EM), can be significantly affected by variations in atmospheric, illumination or environmental conditions typically occurring within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, what propagates significant mismodeling errors throughout t… ▽ More The spectral signatures of the materials contained in hyperspectral images, also called endmembers (EM), can be significantly affected by variations in atmospheric, illumination or environmental conditions typically occurring within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, what propagates significant mismodeling errors throughout the whole unmixing process and compromises the quality of its results. Therefore, large efforts have been recently dedicated to mitigate the effects of spectral variability in SU. This resulted in the development of algorithms that incorporate different strategies to allow the EMs to vary within a hyperspectral image, using, for instance, sets of spectral signatures known a priori, Bayesian, parametric, or local EM models. Each of these approaches has different characteristics and underlying motivations. This paper presents a comprehensive literature review contextualizing both classic and recent approaches to solve this problem. We give a detailed evaluation of the sources of spectral variability and their effect in image spectra. Furthermore, we propose a new taxonomy that organizes existing works according to a practitioner's point of view, based on the necessary amount of supervision and on the computational cost they require. We also review methods used to construct spectral libraries (which are required by many SU techniques) based on the observed hyperspectral image, as well as algorithms for library augmentation and reduction. Finally, we conclude the paper with some discussions and an outline of possible future directions for the field. △ Less

Submitted 6 April, 2021; v1 submitted 20 January, 2020; originally announced January 2020.

Comments: Published at the IEEE Geoscience and Remote Sensing Magazine

Kalman Filtering and Expectation Maximization for Multitemporal Spectral Unmixing

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas, José Carlos Moreira Bermudez, Cédric Richard

Abstract: The recent evolution of hyperspectral imaging technology and the proliferation of new emerging applications presses for the processing of multiple temporal hyperspectral images. In this work, we propose a novel spectral unmixing (SU) strategy using physically motivated parametric endmember representations to account for temporal spectral variability. By representing the multitemporal mixing proces… ▽ More The recent evolution of hyperspectral imaging technology and the proliferation of new emerging applications presses for the processing of multiple temporal hyperspectral images. In this work, we propose a novel spectral unmixing (SU) strategy using physically motivated parametric endmember representations to account for temporal spectral variability. By representing the multitemporal mixing process using a state-space formulation, we are able to exploit the Bayesian filtering machinery to estimate the endmember variability coefficients. Moreover, by assuming that the temporal variability of the abundances is small over short intervals, an efficient implementation of the expectation maximization (EM) algorithm is employed to estimate the abundances and the other model parameters. Simulation results indicate that the proposed strategy outperforms state-of-the-art multitemporal SU algorithms. △ Less

Submitted 12 August, 2020; v1 submitted 2 January, 2020; originally announced January 2020.

A Blind Multiscale Spatial Regularization Framework for Kernel-based Spectral Unmixing

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard

Abstract: Introducing spatial prior information in hyperspectral imaging (HSI) analysis has led to an overall improvement of the performance of many HSI methods applied for denoising, classification, and unmixing. Extending such methodologies to nonlinear settings is not always straightforward, specially for unmixing problems where the consideration of spatial relationships between neighboring pixels might… ▽ More Introducing spatial prior information in hyperspectral imaging (HSI) analysis has led to an overall improvement of the performance of many HSI methods applied for denoising, classification, and unmixing. Extending such methodologies to nonlinear settings is not always straightforward, specially for unmixing problems where the consideration of spatial relationships between neighboring pixels might comprise intricate interactions between their fractional abundances and nonlinear contributions. In this paper, we consider a multiscale regularization strategy for nonlinear spectral unmixing with kernels. The proposed methodology splits the unmixing problem into two sub-problems at two different spatial scales: a coarse scale containing low-dimensional structures, and the original fine scale. The coarse spatial domain is defined using superpixels that result from a multiscale transformation. Spectral unmixing is then formulated as the solution of quadratically constrained optimization problems, which are solved efficiently by exploring their strong duality and a reformulation of their dual cost functions in the form of root-finding problems. Furthermore, we employ a theory-based statistical framework to devise a consistent strategy to estimate all required parameters, including both the regularization parameters of the algorithm and the number of superpixels of the transformation, resulting in a truly blind (from the parameters setting perspective) unmixing method. Experimental results attest the superior performance of the proposed method when comparing with other, state-of-the-art, related strategies. △ Less

Submitted 2 March, 2020; v1 submitted 19 August, 2019; originally announced August 2019.