Showing 1–21 of 21 results for author: Stankovic, L

SAR Despeckling via Regional Denoising Diffusion Probabilistic Model

Authors: Xuran Hu, Ziqiang Xu, Zhihan Chen, Zhengpeng Feng, Mingzhe Zhu, LJubisa Stankovic

Abstract: Speckle noise poses a significant challenge in maintaining the quality of synthetic aperture radar (SAR) images, so SAR despeckling techniques have drawn increasing attention. Despite the tremendous advancements of deep learning in fixed-scale SAR image despeckling, these methods still struggle to deal with large-scale SAR images. To address this problem, this paper introduces a novel despeckling… ▽ More Speckle noise poses a significant challenge in maintaining the quality of synthetic aperture radar (SAR) images, so SAR despeckling techniques have drawn increasing attention. Despite the tremendous advancements of deep learning in fixed-scale SAR image despeckling, these methods still struggle to deal with large-scale SAR images. To address this problem, this paper introduces a novel despeckling approach termed Region Denoising Diffusion Probabilistic Model (R-DDPM) based on generative models. R-DDPM enables versatile despeckling of SAR images across various scales, accomplished within a single training session. Moreover, The artifacts in the fused SAR images can be avoided effectively with the utilization of region-guided inverse sampling. Experiments of our proposed R-DDPM on Sentinel-1 data demonstrates superior performance to existing methods. △ Less

Submitted 5 January, 2024; originally announced January 2024.

Comments: 5 pages, 5 figures

ACM Class: I.4.4

PIPO-Net: A Penalty-based Independent Parameters Optimization Deep Unfolding Network

Authors: Xiumei Li, Zhijie Zhang, Huang Bai, Ljubiša Stanković, Junpeng Hao, Junmei Sun

Abstract: Compressive sensing (CS) has been widely applied in signal and image processing fields. Traditional CS reconstruction algorithms have a complete theoretical foundation but suffer from the high computational complexity, while fashionable deep network-based methods can achieve high-accuracy reconstruction of CS but are short of interpretability. These facts motivate us to develop a deep unfolding ne… ▽ More Compressive sensing (CS) has been widely applied in signal and image processing fields. Traditional CS reconstruction algorithms have a complete theoretical foundation but suffer from the high computational complexity, while fashionable deep network-based methods can achieve high-accuracy reconstruction of CS but are short of interpretability. These facts motivate us to develop a deep unfolding network named the penalty-based independent parameters optimization network (PIPO-Net) to combine the merits of the above mentioned two kinds of CS methods. Each module of PIPO-Net can be viewed separately as an optimization problem with respective penalty function. The main characteristic of PIPO-Net is that, in each round of training, the learnable parameters in one module are updated independently from those of other modules. This makes the network more flexible to find the optimal solutions of the corresponding problems. Moreover, the mean-subtraction sampling and the high-frequency complementary blocks are developed to improve the performance of PIPO-Net. Experiments on reconstructing CS images demonstrate the effectiveness of the proposed PIPO-Net. △ Less

Submitted 4 November, 2023; originally announced November 2023.

Fourier Analysis of Signals on Directed Acyclic Graphs (DAG) Using Graph Zero-Padding

Authors: Ljubisa Stankovic, Milos Dakovic, Ali Bagheri Bardi, Milos Brajovic, Isidora Stankovic

Abstract: Directed acyclic graphs (DAGs) are used for modeling causal relationships, dependencies, and flows in various systems. However, spectral analysis becomes impractical in this setting because the eigen-decomposition of the adjacency matrix yields all eigenvalues equal to zero. This inherent property of DAGs results in an inability to differentiate between frequency components of signals on such grap… ▽ More Directed acyclic graphs (DAGs) are used for modeling causal relationships, dependencies, and flows in various systems. However, spectral analysis becomes impractical in this setting because the eigen-decomposition of the adjacency matrix yields all eigenvalues equal to zero. This inherent property of DAGs results in an inability to differentiate between frequency components of signals on such graphs. This problem can be addressed by alternating the Fourier basis or adding edges in a DAG. However, these approaches change the physics of the considered problem. To address this limitation, we propose a graph zero-padding approach. This approach involves augmenting the original DAG with additional vertices that are connected to the existing structure. The added vertices are characterized by signal values set to zero. The proposed technique enables the spectral evaluation of system outputs on DAGs (in almost all cases), that is the computation of vertex-domain convolution without the adverse effects of aliasing due to changes in a graph structure, with the ultimate goal of preserving the output of the system on a graph as if the changes in the graph structure were not done. △ Less

Submitted 13 November, 2023; v1 submitted 2 November, 2023; originally announced November 2023.

Comments: 10 pages, 12 figures

Eigenvalues of Symmetric Non-normalized Discrete Trigonometric Transforms

Authors: Ali Bagheri Bardi, Milos Dakovic, Taher Yazdanpanah, Ljubisa Stankovic

Abstract: A comprehensive approach to the spectrum characterization (derivation of eigenvalues and the corresponding multiplicities) for non-normalized, symmetric discrete trigonometric transforms (DTT) is presented in the paper. Eight types of the DTT are analyzed. New explicit analytic expressions for the eigenvalues, together with their multiplicities, for the cases of three DTT (DCT$_{(1)}$, DCT… ▽ More A comprehensive approach to the spectrum characterization (derivation of eigenvalues and the corresponding multiplicities) for non-normalized, symmetric discrete trigonometric transforms (DTT) is presented in the paper. Eight types of the DTT are analyzed. New explicit analytic expressions for the eigenvalues, together with their multiplicities, for the cases of three DTT (DCT$_{(1)}$, DCT$_{(5)}$, and DST$_{(8)}$), are the main contribution of this paper. Moreover, the presented theory is supplemented by new, original derivations for the closed-form expressions of the square and the trace of analyzed DTT matrices. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Comments: 9 pages

Demystifying CNNs for Images by Matched Filters

Authors: Shengxi Li, Xinyi Zhao, Ljubisa Stankovic, Danilo Mandic

Abstract: The success of convolution neural networks (CNN) has been revolutionising the way we approach and use intelligent machines in the Big Data era. Despite success, CNNs have been consistently put under scrutiny owing to their \textit{black-box} nature, an \textit{ad hoc} manner of their construction, together with the lack of theoretical support and physical meanings of their operation. This has been… ▽ More The success of convolution neural networks (CNN) has been revolutionising the way we approach and use intelligent machines in the Big Data era. Despite success, CNNs have been consistently put under scrutiny owing to their \textit{black-box} nature, an \textit{ad hoc} manner of their construction, together with the lack of theoretical support and physical meanings of their operation. This has been prohibitive to both the quantitative and qualitative understanding of CNNs, and their application in more sensitive areas such as AI for health. We set out to address these issues, and in this way demystify the operation of CNNs, by employing the perspective of matched filtering. We first illuminate that the convolution operation, the very core of CNNs, represents a matched filter which aims to identify the presence of features in input data. This then serves as a vehicle to interpret the convolution-activation-pooling chain in CNNs under the theoretical umbrella of matched filtering, a common operation in signal processing. We further provide extensive examples and experiments to illustrate this connection, whereby the learning in CNNs is shown to also perform matched filtering, which further sheds light onto physical meaning of learnt parameters and layers. It is our hope that this material will provide new insights into the understanding, constructing and analysing of CNNs, as well as paving the way for develo** new methods and architectures of CNNs. △ Less

Submitted 16 October, 2022; originally announced October 2022.

Analytical Interpretation of Latent Codes in InfoGAN with SAR Images

Authors: Zhenpeng Feng, Milos Dakovic, Hongbing Ji, Mingzhe Zhu, Ljubisa Stankovic

Abstract: Generative Adversarial Networks (GANs) can synthesize abundant photo-realistic synthetic aperture radar (SAR) images. Some recent GANs (e.g., InfoGAN), are even able to edit specific properties of the synthesized images by introducing latent codes. It is crucial for SAR image synthesis since the targets in real SAR images are with different properties due to the imaging mechanism. Despite the succ… ▽ More Generative Adversarial Networks (GANs) can synthesize abundant photo-realistic synthetic aperture radar (SAR) images. Some recent GANs (e.g., InfoGAN), are even able to edit specific properties of the synthesized images by introducing latent codes. It is crucial for SAR image synthesis since the targets in real SAR images are with different properties due to the imaging mechanism. Despite the success of InfoGAN in manipulating properties, there still lacks a clear explanation of how these latent codes affect synthesized properties, thus editing specific properties usually relies on empirical trials, unreliable and time-consuming. In this paper, we show that latent codes are disentangled to affect the properties of SAR images in a non-linear manner. By introducing some property estimators for latent codes, we are able to provide a completely analytical nonlinear model to decompose the entangled causality between latent codes and different properties. The qualitative and quantitative experimental results further reveal that the properties can be calculated by latent codes, inversely, the satisfying latent codes can be estimated given desired properties. In this case, properties can be manipulated by latent codes as we expect. △ Less

Submitted 26 May, 2022; originally announced May 2022.

Comments: 13 pages, 14 figures

How likely is a random graph shift-enabled?

Authors: Liyan Chen, Samuel Cheng, Vladimir Stankovic, Lina Stankovic

Abstract: The shift-enabled property of an underlying graph is essential in designing distributed filters. This article discusses when a random graph is shift-enabled. In particular, popular graph models ER, WS, BA random graph are used, weighted and unweighted, as well as signed graphs. Our results show that the considered unweighted connected random graphs are shift-enabled with high probability when the… ▽ More The shift-enabled property of an underlying graph is essential in designing distributed filters. This article discusses when a random graph is shift-enabled. In particular, popular graph models ER, WS, BA random graph are used, weighted and unweighted, as well as signed graphs. Our results show that the considered unweighted connected random graphs are shift-enabled with high probability when the number of edges is moderately high. However, very dense graphs, as well as fully connected graphs, are not shift-enabled. Interestingly, this behaviour is not observed for weighted connected graphs, which are always shift-enabled unless the number of edges in the graph is very low. △ Less

Submitted 28 August, 2021; originally announced September 2021.

Comments: 9 pages

Dynamic Portfolio Cuts: A Spectral Approach to Graph-Theoretic Diversification

Authors: Alvaro Arroyo, Bruno Scalzo, Ljubisa Stankovic, Danilo P. Mandic

Abstract: Stock market returns are typically analyzed using standard regression, yet they reside on irregular domains which is a natural scenario for graph signal processing. To this end, we consider a market graph as an intuitive way to represent the relationships between financial assets. Traditional methods for estimating asset-return covariance operate under the assumption of statistical time-invariance… ▽ More Stock market returns are typically analyzed using standard regression, yet they reside on irregular domains which is a natural scenario for graph signal processing. To this end, we consider a market graph as an intuitive way to represent the relationships between financial assets. Traditional methods for estimating asset-return covariance operate under the assumption of statistical time-invariance, and are thus unable to appropriately infer the underlying true structure of the market graph. This work introduces a class of graph spectral estimators which cater for the nonstationarity inherent to asset price movements, and serve as a basis to represent the time-varying interactions between assets through a dynamic spectral market graph. Such an account of the time-varying nature of the asset-return covariance allows us to introduce the notion of dynamic spectral portfolio cuts, whereby the graph is partitioned into time-evolving clusters, allowing for online and robust asset allocation. The advantages of the proposed framework over traditional methods are demonstrated through numerical case studies using real-world price data. △ Less

Submitted 7 June, 2021; originally announced June 2021.

Comments: 5 pages, 3 Figures, 2 Tables

Nonstationary Portfolios: Diversification in the Spectral Domain

Authors: Bruno Scalzo, Alvaro Arroyo, Ljubisa Stankovic, Danilo P. Mandic

Abstract: Classical portfolio optimization methods typically determine an optimal capital allocation through the implicit, yet critical, assumption of statistical time-invariance. Such models are inadequate for real-world markets as they employ standard time-averaging based estimators which suffer significant information loss if the market observables are non-stationary. To this end, we reformulate the port… ▽ More Classical portfolio optimization methods typically determine an optimal capital allocation through the implicit, yet critical, assumption of statistical time-invariance. Such models are inadequate for real-world markets as they employ standard time-averaging based estimators which suffer significant information loss if the market observables are non-stationary. To this end, we reformulate the portfolio optimization problem in the spectral domain to cater for the nonstationarity inherent to asset price movements and, in this way, allow for optimal capital allocations to be time-varying. Unlike existing spectral portfolio techniques, the proposed framework employs augmented complex statistics in order to exploit the interactions between the real and imaginary parts of the complex spectral variables, which in turn allows for the modelling of both harmonics and cyclostationarity in the time domain. The advantages of the proposed framework over traditional methods are demonstrated through numerical simulations using real-world price data. △ Less

Submitted 31 January, 2021; originally announced February 2021.

Comments: 5 pages, 3 figures, 1 table. arXiv admin note: text overlap with arXiv:2007.13855

A Probabilistic Spectral Analysis of Multivariate Real-Valued Nonstationary Signals

Authors: Bruno Scalzo, Ljubisa Stankovic, Danilo P. Mandic

Abstract: A class of multivariate spectral representations for real-valued nonstationary random variables is introduced, which is characterised by a general complex Gaussian distribution. In this way, the temporal signal properties -- harmonicity, wide-sense stationarity and cyclostationarity -- are designated respectively by the mean, Hermitian variance and pseudo-variance of the associated time-frequency… ▽ More A class of multivariate spectral representations for real-valued nonstationary random variables is introduced, which is characterised by a general complex Gaussian distribution. In this way, the temporal signal properties -- harmonicity, wide-sense stationarity and cyclostationarity -- are designated respectively by the mean, Hermitian variance and pseudo-variance of the associated time-frequency representation (TFR). For rigour, the estimators of the TFR distribution parameters are derived within a maximum likelihood framework and are shown to be statistically consistent, owing to the statistical identifiability of the proposed distribution parametrization. By virtue of the assumed probabilistic model, a generalised likelihood ratio test (GLRT) for nonstationarity detection is also proposed. Intuitive examples demonstrate the utility of the derived probabilistic framework for spectral analysis in low-SNR environments. △ Less

Submitted 27 July, 2020; originally announced July 2020.

The Support Uncertainty Principle and the Graph Rihaczek Distribution: Revisited and Improved

Authors: Ljubisa Stankovic

Abstract: The classical support uncertainty principle states that the signal and its discrete Fourier transform (DFT) cannot be localized simultaneously in an arbitrary small area in the time and the frequency domain. The product of the number of nonzero samples in the time domain and the frequency domain is greater or equal to the total number of signal samples. The support uncertainty principle has been e… ▽ More The classical support uncertainty principle states that the signal and its discrete Fourier transform (DFT) cannot be localized simultaneously in an arbitrary small area in the time and the frequency domain. The product of the number of nonzero samples in the time domain and the frequency domain is greater or equal to the total number of signal samples. The support uncertainty principle has been extended to the arbitrary orthogonal pairs of signal basis and the graph signals, stating that the product of supports in the vertex domain and the spectral domain is greater than the reciprocal squared maximum absolute value of the basis functions. This form is then used in compressive sensing and sparse signal processing to define the reconstruction conditions. In this paper, we will revisit the graph signal uncertainty principle using the graph Rihaczek distribution as an analysis tool and derive an improved bound for the support uncertainty principle of graph signals. △ Less

Submitted 19 June, 2020; v1 submitted 16 May, 2020; originally announced May 2020.

Comments: 5 pages, 2 figures

Journal ref: IEEE Signal Procesing letters, 2020

RANSAC-Based Signal Denoising Using Compressive Sensing

Authors: Ljubisa Stankovic, Milos Brajovic, Isidora Stankovic, Jonatan Lerga, Milos Dakovic

Abstract: In this paper, we present an approach to the reconstruction of signals exhibiting sparsity in a transformation domain, having some heavily disturbed samples. This sparsity-driven signal recovery exploits a carefully suited random sampling consensus (RANSAC) methodology for the selection of an inlier subset of samples. To this aim, two fundamental properties are used: a signal sample represents a l… ▽ More In this paper, we present an approach to the reconstruction of signals exhibiting sparsity in a transformation domain, having some heavily disturbed samples. This sparsity-driven signal recovery exploits a carefully suited random sampling consensus (RANSAC) methodology for the selection of an inlier subset of samples. To this aim, two fundamental properties are used: a signal sample represents a linear combination of the sparse coefficients, whereas the disturbance degrade original signal sparsity. The properly selected samples are further used as measurements in the sparse signal reconstruction, performed using algorithms from the compressive sensing framework. Besides the fact that the disturbance degrades signal sparsity in the transformation domain, no other disturbance-related assumptions are made -- there are no special requirements regarding its statistical behavior or the range of its values. As a case study, the discrete Fourier transform (DFT) is considered as a domain of signal sparsity, owing to its significance in signal processing theory and applications. Numerical results strongly support the presented theory. In addition, exact relation for the signal-to-noise ratio (SNR) of the reconstructed signal is also presented. This simple result, which conveniently characterizes the RANSAC-based reconstruction performance, is numerically confirmed by a set of statistical examples. △ Less

Submitted 27 March, 2020; originally announced March 2020.

Comments: 4 figures, 15 pages

Graph Signal Processing -- Part III: Machine Learning on Graphs, from Graph Topology to Applications

Authors: Ljubisa Stankovic, Danilo Mandic, Milos Dakovic, Milos Brajovic, Bruno Scalzo, Shengxi Li, Anthony G. Constantinides

Abstract: Many modern data analytics applications on graphs operate on domains where graph topology is not known a priori, and hence its determination becomes part of the problem definition, rather than serving as prior knowledge which aids the problem solution. Part III of this monograph starts by addressing ways to learn graph topology, from the case where the physics of the problem already suggest a poss… ▽ More Many modern data analytics applications on graphs operate on domains where graph topology is not known a priori, and hence its determination becomes part of the problem definition, rather than serving as prior knowledge which aids the problem solution. Part III of this monograph starts by addressing ways to learn graph topology, from the case where the physics of the problem already suggest a possible topology, through to most general cases where the graph topology is learned from the data. A particular emphasis is on graph topology definition based on the correlation and precision matrices of the observed data, combined with additional prior knowledge and structural conditions, such as the smoothness or sparsity of graph connections. For learning sparse graphs (with small number of edges), the least absolute shrinkage and selection operator, known as LASSO is employed, along with its graph specific variant, graphical LASSO. For completeness, both variants of LASSO are derived in an intuitive way, and explained. An in-depth elaboration of the graph topology learning paradigm is provided through several examples on physically well defined graphs, such as electric circuits, linear heat transfer, social and computer networks, and spring-mass systems. As many graph neural networks (GNN) and convolutional graph networks (GCN) are emerging, we have also reviewed the main trends in GNNs and GCNs, from the perspective of graph signal filtering. Tensor representation of lattice-structured graphs is next considered, and it is shown that tensors (multidimensional data arrays) are a special class of graph signals, whereby the graph vertices reside on a high-dimensional regular lattice structure. This part of monograph concludes with two emerging applications in financial data processing and underground transportation networks modeling. △ Less

Submitted 2 January, 2020; originally announced January 2020.

Comments: 61 pages, 55 figures, 40 examples

Portfolio Cuts: A Graph-Theoretic Framework to Diversification

Authors: Bruno Scalzo Dees, Ljubisa Stankovic, Anthony G. Constantinides, Danilo P. Mandic

Abstract: Investment returns naturally reside on irregular domains, however, standard multivariate portfolio optimization methods are agnostic to data structure. To this end, we investigate ways for domain knowledge to be conveniently incorporated into the analysis, by means of graphs. Next, to relax the assumption of the completeness of graph topology and to equip the graph model with practically relevant… ▽ More Investment returns naturally reside on irregular domains, however, standard multivariate portfolio optimization methods are agnostic to data structure. To this end, we investigate ways for domain knowledge to be conveniently incorporated into the analysis, by means of graphs. Next, to relax the assumption of the completeness of graph topology and to equip the graph model with practically relevant physical intuition, we introduce the portfolio cut paradigm. Such a graph-theoretic portfolio partitioning technique is shown to allow the investor to devise robust and tractable asset allocation schemes, by virtue of a rigorous graph framework for considering smaller, computationally feasible, and economically meaningful clusters of assets, based on graph cuts. In turn, this makes it possible to fully utilize the asset returns covariance matrix for constructing the portfolio, even without the requirement for its inversion. The advantages of the proposed framework over traditional methods are demonstrated through numerical simulations based on real-world price data. △ Less

Submitted 16 October, 2019; v1 submitted 12 October, 2019; originally announced October 2019.

Comments: 5 pages, 4 figures

Unitary Shift Operators on a Graph

Authors: Bruno Scalzo Dees, Ljubisa Stankovic, Milos Dakovic, Anthony G. Constantinides, Danilo P. Mandic

Abstract: A unitary shift operator (GSO) for signals on a graph is introduced, which exhibits the desired property of energy preservation over both backward and forward graph shifts. For rigour, the graph differential operator is also derived in an analytical form. The commutativity relation of the shift operator with the Fourier transform is next explored in conjunction with the proposed GSO to introduce a… ▽ More A unitary shift operator (GSO) for signals on a graph is introduced, which exhibits the desired property of energy preservation over both backward and forward graph shifts. For rigour, the graph differential operator is also derived in an analytical form. The commutativity relation of the shift operator with the Fourier transform is next explored in conjunction with the proposed GSO to introduce a graph discrete Fourier transform (GDFT) which, unlike existing approaches, ensures the orthogonality of GDFT bases and admits a natural frequency-domain interpretation. The proposed GDFT is shown to allow for a coherent definition of the graph discrete Hilbert transform (GDHT) and the graph analytic signal. The advantages of the proposed GSO are demonstrated through illustrative examples. △ Less

Submitted 17 September, 2019; v1 submitted 12 September, 2019; originally announced September 2019.

Comments: 5 pages, 3 figures

A Class of Doubly Stochastic Shift Operators for Random Graph Signals and their Boundedness

Authors: Bruno Scalzo Dees, Ljubisa Stankovic, Milos Dakovic, Anthony G. Constantinides, Danilo P. Mandic

Abstract: A class of doubly stochastic graph shift operators (GSO) is proposed, which is shown to exhibit: (i) lower and upper $L_{2}$-boundedness for locally stationary random graph signals; (ii) $L_{2}$-isometry for \textit{i.i.d.} random graph signals with the asymptotic increase in the incoming neighbourhood size of vertices; and (iii) preservation of the mean of any graph signal. These properties are o… ▽ More A class of doubly stochastic graph shift operators (GSO) is proposed, which is shown to exhibit: (i) lower and upper $L_{2}$-boundedness for locally stationary random graph signals; (ii) $L_{2}$-isometry for \textit{i.i.d.} random graph signals with the asymptotic increase in the incoming neighbourhood size of vertices; and (iii) preservation of the mean of any graph signal. These properties are obtained through a statistical consistency analysis of the graph shift, and by exploiting the dual role of the doubly stochastic GSO as a Markov (diffusion) matrix and as an unbiased expectation operator. Practical utility of the class of doubly stochastic GSOs is demonstrated in a real-world multi-sensor signal filtering setting. △ Less

Submitted 7 February, 2020; v1 submitted 5 August, 2019; originally announced August 2019.

Comments: 5 pages, 1 figure

Vertex-Frequency Graph Signal Processing: A review

Authors: Ljubisa Stankovic, Danilo P. Mandic, Milos Dakovic, Bruno Scalzo, Milos Brajovic, Ervin Sejdic, Anthony G. Constantinides

Abstract: Graph signal processing deals with signals which are observed on an irregular graph domain. While many approaches have been developed in classical graph theory to cluster vertices and segment large graphs in a signal independent way, signal localization based approaches to the analysis of data on graph represent a new research direction which is also a key to big data analytics on graphs. To this… ▽ More Graph signal processing deals with signals which are observed on an irregular graph domain. While many approaches have been developed in classical graph theory to cluster vertices and segment large graphs in a signal independent way, signal localization based approaches to the analysis of data on graph represent a new research direction which is also a key to big data analytics on graphs. To this end, after an overview of the basic definitions in graphs and graph signals, we present and discuss a localized form of the graph Fourier transform. To establish an analogy with classical signal processing, spectral- and vertex-domain definitions of the localization window are given next. The spectral and vertex localization kernels are then related to the wavelet transform, followed by a study of filtering and inversion of the localized graph Fourier transform. For rigour, the analysis of energy representation and frames in the localized graph Fourier transform is extended to the energy forms of vertex-frequency distributions, which operate even without the need to apply localization windows. Another link with classical signal processing is established through the concept of local smoothness, which is subsequently related to the particular paradigm of signal smoothness on graphs. This all represents a comprehensive account of the relation of general vertex-frequency analysis with classical time-frequency analysis, and important but missing link for more advanced applications of graphs signal processing. The theory is supported by illustrative and practically relevant examples. △ Less

Submitted 26 December, 2019; v1 submitted 8 July, 2019; originally announced July 2019.

Comments: 16 pages, 12 figures

Graph Signal Processing -- Part I: Graphs, Graph Spectra, and Spectral Clustering

Authors: Ljubisa Stankovic, Danilo Mandic, Milos Dakovic, Milos Brajovic, Bruno Scalzo, Tony Constantinides

Abstract: The area of Data Analytics on graphs promises a paradigm shift as we approach information processing of classes of data, which are typically acquired on irregular but structured domains (social networks, various ad-hoc sensor networks). Yet, despite its long history, current approaches mostly focus on the optimization of graphs themselves, rather than on directly inferring learning strategies, suc… ▽ More The area of Data Analytics on graphs promises a paradigm shift as we approach information processing of classes of data, which are typically acquired on irregular but structured domains (social networks, various ad-hoc sensor networks). Yet, despite its long history, current approaches mostly focus on the optimization of graphs themselves, rather than on directly inferring learning strategies, such as detection, estimation, statistical and probabilistic inference, clustering and separation from signals and data acquired on graphs. To fill this void, we first revisit graph topologies from a Data Analytics point of view, and establish a taxonomy of graph networks through a linear algebraic formalism of graph topology (vertices, connections, directivity). This serves as a basis for spectral analysis of graphs, whereby the eigenvalues and eigenvectors of graph Laplacian and adjacency matrices are shown to convey physical meaning related to both graph topology and higher-order graph properties, such as cuts, walks, paths, and neighborhoods. Next, to illustrate estimation strategies performed on graph signals, spectral analysis of graphs is introduced through eigenanalysis of mathematical descriptors of graphs and in a generic way. Finally, a framework for vertex clustering and graph segmentation is established based on graph spectral representation (eigenanalysis) which illustrates the power of graphs in various data association tasks. The supporting examples demonstrate the promise of Graph Data Analytics in modeling structural and functional/semantic inferences. At the same time, Part I serves as a basis for Part II and Part III which deal with theory, methods and applications of processing Data on Graphs and Graph Topology Learning from data. △ Less

Submitted 12 August, 2019; v1 submitted 8 July, 2019; originally announced July 2019.

Comments: 49 pages, 40 figures

An Example-Driven Introduction to Data Analytics on Graphs

Authors: Ljubisa Stankovic, Danilo Mandic, Milos Dakovic, Ilya Kisil, Ervin Sejdic, Anthony G. Constantinides

Abstract: Graphs are irregular structures which naturally account for data integrity, however, traditional approaches have been established outside Signal Processing, and largely focus on analyzing the underlying graphs rather than signals on graphs. Given the rapidly increasing availability of multisensor and multinode measurements, likely recorded on irregular or ad-hoc grids, it would be extremely advant… ▽ More Graphs are irregular structures which naturally account for data integrity, however, traditional approaches have been established outside Signal Processing, and largely focus on analyzing the underlying graphs rather than signals on graphs. Given the rapidly increasing availability of multisensor and multinode measurements, likely recorded on irregular or ad-hoc grids, it would be extremely advantageous to analyze such structured data as graph signals and thus benefit from the ability of graphs to incorporate spatial awareness of the sensing locations, sensor importance, and local versus global sensor association. The aim of this lecture note is therefore to establish a common language between graph signals, defined on irregular signal domains, and some of the most fundamental paradigms in DSP, such as spectral analysis of multichannel signals, system transfer function, digital filter design, parameter estimation, and optimal filter design. This is achieved through a physically meaningful and intuitive real-world example of geographically distributed multisensor temperature estimation. A similar spatial multisensor arrangement is already widely used in Signal Processing curricula to introduce minimum variance estimators and Kalman filters \cite{HM}, and by adopting this framework we facilitate a seamless integration of graph theory into the curriculum of existing DSP courses. By bridging the gap between standard approaches and graph signal processing, we also show that standard methods can be thought of as special cases of their graph counterparts, evaluated on linear graphs. It is hoped that our approach would not only help to demystify graph theoretic approaches in education and research but it would also empower practitioners to explore a whole host of otherwise prohibitive modern applications. △ Less

Submitted 12 May, 2019; v1 submitted 26 March, 2019; originally announced March 2019.

Comments: 10 pages, 3 figures, 5 boxes

Undirected graphs: is the shift-enabled condition trivial or necessary?

Authors: Liyan Chen, Samuel Cheng, Kanghang He, Lina Stankovic, Vladimir Stankovic

Abstract: It has recently been shown that, contrary to the wide belief that a shift-enabled condition (necessary for any shift-invariant filter to be representable by a graph shift matrix) can be ignored because any non-shift-enabled matrix can be converted to a shift-enabled matrix, such a conversion in general may not hold for a directed graph with non-symmetric shift matrix. This letter extends this prio… ▽ More It has recently been shown that, contrary to the wide belief that a shift-enabled condition (necessary for any shift-invariant filter to be representable by a graph shift matrix) can be ignored because any non-shift-enabled matrix can be converted to a shift-enabled matrix, such a conversion in general may not hold for a directed graph with non-symmetric shift matrix. This letter extends this prior work, focusing on undirected graphs where the shift matrix is generally symmetric. We show that while, in this case, the shift matrix can be converted to satisfy the original shift-enabled condition, the converted matrix is not associated with the original graph, that is, it does not capture anymore the structure of the graph signal. We show via a counterexample, that a non-shift-enabled matrix cannot be converted to a shift-enabled one and still maintain the topological structure of the underlying graph, which is necessary to facilitate localized signal processing. △ Less

Submitted 22 May, 2019; v1 submitted 30 October, 2018; originally announced October 2018.

Comments: 5 pages, 1 figures

Shift-enabled graphs: Graphs where shift-invariant filters are representable as polynomials of shift operations

Authors: Liyan Chen, Samuel Cheng, Vlandimir Stankovic, Lina Stankovic

Abstract: In digital signal processing, shift-invariant filters can be represented as a polynomial expansion of a shift operation,that is, the Z-transform representation. When extended to graph signal processing (GSP), this would mean that a shift-invariant graph filter can be represented as a polynomial of the adjacency (shift) matrix of the graph. However, the characteristic and minimum polynomials of the… ▽ More In digital signal processing, shift-invariant filters can be represented as a polynomial expansion of a shift operation,that is, the Z-transform representation. When extended to graph signal processing (GSP), this would mean that a shift-invariant graph filter can be represented as a polynomial of the adjacency (shift) matrix of the graph. However, the characteristic and minimum polynomials of the adjacency matrix must be identical for the property to hold. While it has been suggested that this condition might be ignored as it is always possible to find a polynomial transform to represent the original adjacency matrix by another adjacency matrix that satisfies the condition, this letter shows that a filter that is shift invariant in terms of the original graph may not be shift invariant anymore under the modified graph and vice versa. We introduce the notion of "shift-enabled graph" for graphs that satisfy the aforementioned condition, and present a concrete example of a graph that is not "shift-enabled" and a shift-invariant filter that is not a polynomial of the shift operation matrix. The result provides a deeper understanding of shift-invariant filters when applied in GSP and shows that further investigation of shift-enabled graphs is needed to make it applicable to practical scenarios. △ Less

Submitted 10 June, 2018; v1 submitted 28 October, 2017; originally announced October 2017.

Comments: 4 pages, 3 figures