-
Fourier Analysis of Signals on Directed Acyclic Graphs (DAG) Using Graph Zero-Padding
Authors:
Ljubisa Stankovic,
Milos Dakovic,
Ali Bagheri Bardi,
Milos Brajovic,
Isidora Stankovic
Abstract:
Directed acyclic graphs (DAGs) are used for modeling causal relationships, dependencies, and flows in various systems. However, spectral analysis becomes impractical in this setting because the eigen-decomposition of the adjacency matrix yields all eigenvalues equal to zero. This inherent property of DAGs results in an inability to differentiate between frequency components of signals on such grap…
▽ More
Directed acyclic graphs (DAGs) are used for modeling causal relationships, dependencies, and flows in various systems. However, spectral analysis becomes impractical in this setting because the eigen-decomposition of the adjacency matrix yields all eigenvalues equal to zero. This inherent property of DAGs results in an inability to differentiate between frequency components of signals on such graphs. This problem can be addressed by alternating the Fourier basis or adding edges in a DAG. However, these approaches change the physics of the considered problem. To address this limitation, we propose a graph zero-padding approach. This approach involves augmenting the original DAG with additional vertices that are connected to the existing structure. The added vertices are characterized by signal values set to zero. The proposed technique enables the spectral evaluation of system outputs on DAGs (in almost all cases), that is the computation of vertex-domain convolution without the adverse effects of aliasing due to changes in a graph structure, with the ultimate goal of preserving the output of the system on a graph as if the changes in the graph structure were not done.
△ Less
Submitted 13 November, 2023; v1 submitted 2 November, 2023;
originally announced November 2023.
-
Eigenvalues of Symmetric Non-normalized Discrete Trigonometric Transforms
Authors:
Ali Bagheri Bardi,
Milos Dakovic,
Taher Yazdanpanah,
Ljubisa Stankovic
Abstract:
A comprehensive approach to the spectrum characterization (derivation of eigenvalues and the corresponding multiplicities) for non-normalized, symmetric discrete trigonometric transforms (DTT) is presented in the paper. Eight types of the DTT are analyzed. New explicit analytic expressions for the eigenvalues, together with their multiplicities, for the cases of three DTT (DCT$_{(1)}$, DCT…
▽ More
A comprehensive approach to the spectrum characterization (derivation of eigenvalues and the corresponding multiplicities) for non-normalized, symmetric discrete trigonometric transforms (DTT) is presented in the paper. Eight types of the DTT are analyzed. New explicit analytic expressions for the eigenvalues, together with their multiplicities, for the cases of three DTT (DCT$_{(1)}$, DCT$_{(5)}$, and DST$_{(8)}$), are the main contribution of this paper. Moreover, the presented theory is supplemented by new, original derivations for the closed-form expressions of the square and the trace of analyzed DTT matrices.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
Cluster-CAM: Cluster-Weighted Visual Interpretation of CNNs' Decision in Image Classification
Authors:
Zhenpeng Feng,
Hongbing Ji,
Milos Dakovic,
Xiyang Cui,
Mingzhe Zhu,
Ljubisa Stankovic
Abstract:
Despite the tremendous success of convolutional neural networks (CNNs) in computer vision, the mechanism of CNNs still lacks clear interpretation. Currently, class activation map** (CAM), a famous visualization technique to interpret CNN's decision, has drawn increasing attention. Gradient-based CAMs are efficient while the performance is heavily affected by gradient vanishing and exploding. In…
▽ More
Despite the tremendous success of convolutional neural networks (CNNs) in computer vision, the mechanism of CNNs still lacks clear interpretation. Currently, class activation map** (CAM), a famous visualization technique to interpret CNN's decision, has drawn increasing attention. Gradient-based CAMs are efficient while the performance is heavily affected by gradient vanishing and exploding. In contrast, gradient-free CAMs can avoid computing gradients to produce more understandable results. However, existing gradient-free CAMs are quite time-consuming because hundreds of forward interference per image are required. In this paper, we proposed Cluster-CAM, an effective and efficient gradient-free CNN interpretation algorithm. Cluster-CAM can significantly reduce the times of forward propagation by splitting the feature maps into clusters in an unsupervised manner. Furthermore, we propose an artful strategy to forge a cognition-base map and cognition-scissors from clustered feature maps. The final salience heatmap will be computed by merging the above cognition maps. Qualitative results conspicuously show that Cluster-CAM can produce heatmaps where the highlighted regions match the human's cognition more precisely than existing CAMs. The quantitative evaluation further demonstrates the superiority of Cluster-CAM in both effectiveness and efficiency.
△ Less
Submitted 3 February, 2023;
originally announced February 2023.
-
Analytical Interpretation of Latent Codes in InfoGAN with SAR Images
Authors:
Zhenpeng Feng,
Milos Dakovic,
Hongbing Ji,
Mingzhe Zhu,
Ljubisa Stankovic
Abstract:
Generative Adversarial Networks (GANs) can synthesize abundant photo-realistic synthetic aperture radar (SAR) images. Some recent GANs (e.g., InfoGAN), are even able to edit specific properties of the synthesized images by introducing latent codes. It is crucial for SAR image synthesis since the targets in real SAR images are with different properties due to the imaging mechanism. Despite the succ…
▽ More
Generative Adversarial Networks (GANs) can synthesize abundant photo-realistic synthetic aperture radar (SAR) images. Some recent GANs (e.g., InfoGAN), are even able to edit specific properties of the synthesized images by introducing latent codes. It is crucial for SAR image synthesis since the targets in real SAR images are with different properties due to the imaging mechanism. Despite the success of InfoGAN in manipulating properties, there still lacks a clear explanation of how these latent codes affect synthesized properties, thus editing specific properties usually relies on empirical trials, unreliable and time-consuming. In this paper, we show that latent codes are disentangled to affect the properties of SAR images in a non-linear manner. By introducing some property estimators for latent codes, we are able to provide a completely analytical nonlinear model to decompose the entangled causality between latent codes and different properties. The qualitative and quantitative experimental results further reveal that the properties can be calculated by latent codes, inversely, the satisfying latent codes can be estimated given desired properties. In this case, properties can be manipulated by latent codes as we expect.
△ Less
Submitted 26 May, 2022;
originally announced May 2022.
-
Spontaneous Symmetry Breaking: The Case of Crazy Clock and Beyond
Authors:
Maja C. Pagnacco,
Jelena P. Maksimović,
Marko Daković,
Bojana Bokic,
Sébastien R. Mouchet,
Thierry Verbiest,
Yves Caudano,
Branko Kolaric
Abstract:
In this account, we describe the crazy-clock phenomenon involving the state I (low iodide and iodine concentration) to state II (high iodide and iodine concentration with new iodine phase) transition after a Briggs-Rauscher (BR) oscillatory process. While the BR crazy-clock phenomenon is known, it is the first time that crazy-clock behavior is linked and explained with the symmetry-breaking phenom…
▽ More
In this account, we describe the crazy-clock phenomenon involving the state I (low iodide and iodine concentration) to state II (high iodide and iodine concentration with new iodine phase) transition after a Briggs-Rauscher (BR) oscillatory process. While the BR crazy-clock phenomenon is known, it is the first time that crazy-clock behavior is linked and explained with the symmetry-breaking phenomenon, highlighting the entire process in a novel way. The presented phenomenon has been thoroughly investigated by running more than 60 experiments, and evaluated by using statistical cluster K-means analysis. The mixing rate, as well as the magnetic bar shape and dimensions, have a strong influence on the transition appearance. Although the transition for both mixing and no-mixing conditions are taking place completely randomly, by using statistical cluster analysis we obtain different numbers of clusters (showing the time-domains where the transition is more likely to occur). In the case of stirring, clusters are more compact and separated, revealed new hidden details regarding the chemical dynamics of nonlinear processes. The significance of the presented results is beyond oscillatory reaction kinetics since the described example belongs to the small class of chemical systems that shows intrinsic randomness in their response and it might be considered as a real example of a classical liquid random number generator.
△ Less
Submitted 10 January, 2022;
originally announced January 2022.
-
Improved Coherence Index-Based Bound in Compressive Sensing
Authors:
Ljubisa Stankovic,
Milos Brajovic,
Danilo Mandic,
Isidora Stankovic,
Milos Dakovic
Abstract:
Within the Compressive Sensing (CS) paradigm, sparse signals can be reconstructed based on a reduced set of measurements. Reliability of the solution is determined by the uniqueness condition. With its mathematically tractable and feasible calculation, coherence index is one of very few CS metrics with a considerable practical importance. In this paper, we propose an improvement of the coherence b…
▽ More
Within the Compressive Sensing (CS) paradigm, sparse signals can be reconstructed based on a reduced set of measurements. Reliability of the solution is determined by the uniqueness condition. With its mathematically tractable and feasible calculation, coherence index is one of very few CS metrics with a considerable practical importance. In this paper, we propose an improvement of the coherence based uniqueness relation for the matching pursuit algorithms. Starting from a simple and intuitive derivation of the standard uniqueness condition based on the coherence index, we derive a less conservative coherence index-based lower bound for signal sparsity. The results are generalized to the uniqueness condition of the $l_0$-norm minimization for a signal represented in two orthonormal bases.
△ Less
Submitted 11 March, 2021;
originally announced March 2021.
-
RANSAC-Based Signal Denoising Using Compressive Sensing
Authors:
Ljubisa Stankovic,
Milos Brajovic,
Isidora Stankovic,
Jonatan Lerga,
Milos Dakovic
Abstract:
In this paper, we present an approach to the reconstruction of signals exhibiting sparsity in a transformation domain, having some heavily disturbed samples. This sparsity-driven signal recovery exploits a carefully suited random sampling consensus (RANSAC) methodology for the selection of an inlier subset of samples. To this aim, two fundamental properties are used: a signal sample represents a l…
▽ More
In this paper, we present an approach to the reconstruction of signals exhibiting sparsity in a transformation domain, having some heavily disturbed samples. This sparsity-driven signal recovery exploits a carefully suited random sampling consensus (RANSAC) methodology for the selection of an inlier subset of samples. To this aim, two fundamental properties are used: a signal sample represents a linear combination of the sparse coefficients, whereas the disturbance degrade original signal sparsity. The properly selected samples are further used as measurements in the sparse signal reconstruction, performed using algorithms from the compressive sensing framework. Besides the fact that the disturbance degrades signal sparsity in the transformation domain, no other disturbance-related assumptions are made -- there are no special requirements regarding its statistical behavior or the range of its values. As a case study, the discrete Fourier transform (DFT) is considered as a domain of signal sparsity, owing to its significance in signal processing theory and applications. Numerical results strongly support the presented theory. In addition, exact relation for the signal-to-noise ratio (SNR) of the reconstructed signal is also presented. This simple result, which conveniently characterizes the RANSAC-based reconstruction performance, is numerically confirmed by a set of statistical examples.
△ Less
Submitted 27 March, 2020;
originally announced March 2020.
-
Graph Signal Processing -- Part III: Machine Learning on Graphs, from Graph Topology to Applications
Authors:
Ljubisa Stankovic,
Danilo Mandic,
Milos Dakovic,
Milos Brajovic,
Bruno Scalzo,
Shengxi Li,
Anthony G. Constantinides
Abstract:
Many modern data analytics applications on graphs operate on domains where graph topology is not known a priori, and hence its determination becomes part of the problem definition, rather than serving as prior knowledge which aids the problem solution. Part III of this monograph starts by addressing ways to learn graph topology, from the case where the physics of the problem already suggest a poss…
▽ More
Many modern data analytics applications on graphs operate on domains where graph topology is not known a priori, and hence its determination becomes part of the problem definition, rather than serving as prior knowledge which aids the problem solution. Part III of this monograph starts by addressing ways to learn graph topology, from the case where the physics of the problem already suggest a possible topology, through to most general cases where the graph topology is learned from the data. A particular emphasis is on graph topology definition based on the correlation and precision matrices of the observed data, combined with additional prior knowledge and structural conditions, such as the smoothness or sparsity of graph connections. For learning sparse graphs (with small number of edges), the least absolute shrinkage and selection operator, known as LASSO is employed, along with its graph specific variant, graphical LASSO. For completeness, both variants of LASSO are derived in an intuitive way, and explained. An in-depth elaboration of the graph topology learning paradigm is provided through several examples on physically well defined graphs, such as electric circuits, linear heat transfer, social and computer networks, and spring-mass systems. As many graph neural networks (GNN) and convolutional graph networks (GCN) are emerging, we have also reviewed the main trends in GNNs and GCNs, from the perspective of graph signal filtering. Tensor representation of lattice-structured graphs is next considered, and it is shown that tensors (multidimensional data arrays) are a special class of graph signals, whereby the graph vertices reside on a high-dimensional regular lattice structure. This part of monograph concludes with two emerging applications in financial data processing and underground transportation networks modeling.
△ Less
Submitted 2 January, 2020;
originally announced January 2020.
-
Graph Signal Processing -- Part II: Processing and Analyzing Signals on Graphs
Authors:
Ljubisa Stankovic,
Danilo Mandic,
Milos Dakovic,
Milos Brajovic,
Bruno Scalzo,
Anthony G. Constantinides
Abstract:
The focus of Part I of this monograph has been on both the fundamental properties, graph topologies, and spectral representations of graphs. Part II embarks on these concepts to address the algorithmic and practical issues centered round data/signal processing on graphs, that is, the focus is on the analysis and estimation of both deterministic and random data on graphs. The fundamental ideas rela…
▽ More
The focus of Part I of this monograph has been on both the fundamental properties, graph topologies, and spectral representations of graphs. Part II embarks on these concepts to address the algorithmic and practical issues centered round data/signal processing on graphs, that is, the focus is on the analysis and estimation of both deterministic and random data on graphs. The fundamental ideas related to graph signals are introduced through a simple and intuitive, yet illustrative and general enough case study of multisensor temperature field estimation. The concept of systems on graph is defined using graph signal shift operators, which generalize the corresponding principles from traditional learning systems. At the core of the spectral domain representation of graph signals and systems is the Graph Discrete Fourier Transform (GDFT). The spectral domain representations are then used as the basis to introduce graph signal filtering concepts and address their design, including Chebyshev polynomial approximation series. Ideas related to the sampling of graph signals are presented and further linked with compressive sensing. Localized graph signal analysis in the joint vertex-spectral domain is referred to as the vertex-frequency analysis, since it can be considered as an extension of classical time-frequency analysis to the graph domain of a signal. Important topics related to the local graph Fourier transform (LGFT) are covered, together with its various forms including the graph spectral and vertex domain windows and the inversion conditions and relations. A link between the LGFT with spectral varying window and the spectral graph wavelet transform (SGWT) is also established. Realizations of the LGFT and SGWT using polynomial (Chebyshev) approximations of the spectral functions are further considered. Finally, energy versions of the vertex-frequency representations are introduced.
△ Less
Submitted 23 September, 2019;
originally announced September 2019.
-
Unitary Shift Operators on a Graph
Authors:
Bruno Scalzo Dees,
Ljubisa Stankovic,
Milos Dakovic,
Anthony G. Constantinides,
Danilo P. Mandic
Abstract:
A unitary shift operator (GSO) for signals on a graph is introduced, which exhibits the desired property of energy preservation over both backward and forward graph shifts. For rigour, the graph differential operator is also derived in an analytical form. The commutativity relation of the shift operator with the Fourier transform is next explored in conjunction with the proposed GSO to introduce a…
▽ More
A unitary shift operator (GSO) for signals on a graph is introduced, which exhibits the desired property of energy preservation over both backward and forward graph shifts. For rigour, the graph differential operator is also derived in an analytical form. The commutativity relation of the shift operator with the Fourier transform is next explored in conjunction with the proposed GSO to introduce a graph discrete Fourier transform (GDFT) which, unlike existing approaches, ensures the orthogonality of GDFT bases and admits a natural frequency-domain interpretation. The proposed GDFT is shown to allow for a coherent definition of the graph discrete Hilbert transform (GDHT) and the graph analytic signal. The advantages of the proposed GSO are demonstrated through illustrative examples.
△ Less
Submitted 17 September, 2019; v1 submitted 12 September, 2019;
originally announced September 2019.
-
A Class of Doubly Stochastic Shift Operators for Random Graph Signals and their Boundedness
Authors:
Bruno Scalzo Dees,
Ljubisa Stankovic,
Milos Dakovic,
Anthony G. Constantinides,
Danilo P. Mandic
Abstract:
A class of doubly stochastic graph shift operators (GSO) is proposed, which is shown to exhibit: (i) lower and upper $L_{2}$-boundedness for locally stationary random graph signals; (ii) $L_{2}$-isometry for \textit{i.i.d.} random graph signals with the asymptotic increase in the incoming neighbourhood size of vertices; and (iii) preservation of the mean of any graph signal. These properties are o…
▽ More
A class of doubly stochastic graph shift operators (GSO) is proposed, which is shown to exhibit: (i) lower and upper $L_{2}$-boundedness for locally stationary random graph signals; (ii) $L_{2}$-isometry for \textit{i.i.d.} random graph signals with the asymptotic increase in the incoming neighbourhood size of vertices; and (iii) preservation of the mean of any graph signal. These properties are obtained through a statistical consistency analysis of the graph shift, and by exploiting the dual role of the doubly stochastic GSO as a Markov (diffusion) matrix and as an unbiased expectation operator. Practical utility of the class of doubly stochastic GSOs is demonstrated in a real-world multi-sensor signal filtering setting.
△ Less
Submitted 7 February, 2020; v1 submitted 5 August, 2019;
originally announced August 2019.
-
Vertex-Frequency Graph Signal Processing: A review
Authors:
Ljubisa Stankovic,
Danilo P. Mandic,
Milos Dakovic,
Bruno Scalzo,
Milos Brajovic,
Ervin Sejdic,
Anthony G. Constantinides
Abstract:
Graph signal processing deals with signals which are observed on an irregular graph domain. While many approaches have been developed in classical graph theory to cluster vertices and segment large graphs in a signal independent way, signal localization based approaches to the analysis of data on graph represent a new research direction which is also a key to big data analytics on graphs. To this…
▽ More
Graph signal processing deals with signals which are observed on an irregular graph domain. While many approaches have been developed in classical graph theory to cluster vertices and segment large graphs in a signal independent way, signal localization based approaches to the analysis of data on graph represent a new research direction which is also a key to big data analytics on graphs. To this end, after an overview of the basic definitions in graphs and graph signals, we present and discuss a localized form of the graph Fourier transform. To establish an analogy with classical signal processing, spectral- and vertex-domain definitions of the localization window are given next. The spectral and vertex localization kernels are then related to the wavelet transform, followed by a study of filtering and inversion of the localized graph Fourier transform. For rigour, the analysis of energy representation and frames in the localized graph Fourier transform is extended to the energy forms of vertex-frequency distributions, which operate even without the need to apply localization windows. Another link with classical signal processing is established through the concept of local smoothness, which is subsequently related to the particular paradigm of signal smoothness on graphs. This all represents a comprehensive account of the relation of general vertex-frequency analysis with classical time-frequency analysis, and important but missing link for more advanced applications of graphs signal processing. The theory is supported by illustrative and practically relevant examples.
△ Less
Submitted 26 December, 2019; v1 submitted 8 July, 2019;
originally announced July 2019.
-
Graph Signal Processing -- Part I: Graphs, Graph Spectra, and Spectral Clustering
Authors:
Ljubisa Stankovic,
Danilo Mandic,
Milos Dakovic,
Milos Brajovic,
Bruno Scalzo,
Tony Constantinides
Abstract:
The area of Data Analytics on graphs promises a paradigm shift as we approach information processing of classes of data, which are typically acquired on irregular but structured domains (social networks, various ad-hoc sensor networks). Yet, despite its long history, current approaches mostly focus on the optimization of graphs themselves, rather than on directly inferring learning strategies, suc…
▽ More
The area of Data Analytics on graphs promises a paradigm shift as we approach information processing of classes of data, which are typically acquired on irregular but structured domains (social networks, various ad-hoc sensor networks). Yet, despite its long history, current approaches mostly focus on the optimization of graphs themselves, rather than on directly inferring learning strategies, such as detection, estimation, statistical and probabilistic inference, clustering and separation from signals and data acquired on graphs. To fill this void, we first revisit graph topologies from a Data Analytics point of view, and establish a taxonomy of graph networks through a linear algebraic formalism of graph topology (vertices, connections, directivity). This serves as a basis for spectral analysis of graphs, whereby the eigenvalues and eigenvectors of graph Laplacian and adjacency matrices are shown to convey physical meaning related to both graph topology and higher-order graph properties, such as cuts, walks, paths, and neighborhoods. Next, to illustrate estimation strategies performed on graph signals, spectral analysis of graphs is introduced through eigenanalysis of mathematical descriptors of graphs and in a generic way. Finally, a framework for vertex clustering and graph segmentation is established based on graph spectral representation (eigenanalysis) which illustrates the power of graphs in various data association tasks. The supporting examples demonstrate the promise of Graph Data Analytics in modeling structural and functional/semantic inferences. At the same time, Part I serves as a basis for Part II and Part III which deal with theory, methods and applications of processing Data on Graphs and Graph Topology Learning from data.
△ Less
Submitted 12 August, 2019; v1 submitted 8 July, 2019;
originally announced July 2019.
-
Quantization in Compressive Sensing: A Signal Processing Approach
Authors:
Isidora Stankovic,
Milos Brajovic,
Milos Dakovic,
Cornel Ioana,
Ljubisa Stankovic
Abstract:
Influence of the finite-length registers and quantization effects on the reconstruction of sparse and approximately sparse signals is analyzed in this paper. For the nonquantized measurements, the compressive sensing (CS) framework provides highly accurate reconstruction algorithms that produce negligible errors when the reconstruction conditions are met. However, hardware implementations of signa…
▽ More
Influence of the finite-length registers and quantization effects on the reconstruction of sparse and approximately sparse signals is analyzed in this paper. For the nonquantized measurements, the compressive sensing (CS) framework provides highly accurate reconstruction algorithms that produce negligible errors when the reconstruction conditions are met. However, hardware implementations of signal processing algorithms involve the finite-length registers and quantization of the measurements. An analysis of the effects related to the measurements quantization with an arbitrary number of bits is the topic of this paper. A unified mathematical model for the analysis of the quantization noise and the signal nonsparsity on the CS reconstruction is presented. An exact formula for the expected energy of error in the CS-based reconstructed signal is derived. The theory is validated through various numerical examples with quantized measurements, including the cases of approximately sparse signals, noise folding, and floating-point arithmetics.
△ Less
Submitted 1 July, 2019;
originally announced July 2019.
-
On the Decomposition of Multivariate Nonstationary Multicomponent Signals
Authors:
Ljubisa Stankovic,
Milos Brajovic,
Milos Dakovic,
Danilo Mandic
Abstract:
With their ability to handle an increased amount of information, multivariate and multichannel signals can be used to solve problems normally not solvable with signals obtained from a single source. One such problem is the decomposition signals with several components whose domains of support significantly overlap in both the time and the frequency domain, including the joint time-frequency domain…
▽ More
With their ability to handle an increased amount of information, multivariate and multichannel signals can be used to solve problems normally not solvable with signals obtained from a single source. One such problem is the decomposition signals with several components whose domains of support significantly overlap in both the time and the frequency domain, including the joint time-frequency domain. Initially, we proposed a solution to this problem based on the Wigner distribution of multivariate signals, which requires the attenuation of the cross-terms. In this paper, an advanced solution based on an eigenvalue analysis of the multivariate signal autocorrelation matrix, followed by their time-frequency concentration measure minimization, is presented. This analysis provides less restrictive conditions for the signal decomposition than in the case of Wigner distribution. The algorithm for the components separation is based on the concentration measures of the eigenvector time-frequency representation, that are linear combinations of the overlap** signal components. With an increased number of sensors/channels, the robustness of the decomposition process to additive noise is also achieved. The theory is supported by numerical examples. The required channel dissimilarity is statistically investigated as well.
△ Less
Submitted 31 March, 2019;
originally announced April 2019.
-
An Example-Driven Introduction to Data Analytics on Graphs
Authors:
Ljubisa Stankovic,
Danilo Mandic,
Milos Dakovic,
Ilya Kisil,
Ervin Sejdic,
Anthony G. Constantinides
Abstract:
Graphs are irregular structures which naturally account for data integrity, however, traditional approaches have been established outside Signal Processing, and largely focus on analyzing the underlying graphs rather than signals on graphs. Given the rapidly increasing availability of multisensor and multinode measurements, likely recorded on irregular or ad-hoc grids, it would be extremely advant…
▽ More
Graphs are irregular structures which naturally account for data integrity, however, traditional approaches have been established outside Signal Processing, and largely focus on analyzing the underlying graphs rather than signals on graphs. Given the rapidly increasing availability of multisensor and multinode measurements, likely recorded on irregular or ad-hoc grids, it would be extremely advantageous to analyze such structured data as graph signals and thus benefit from the ability of graphs to incorporate spatial awareness of the sensing locations, sensor importance, and local versus global sensor association. The aim of this lecture note is therefore to establish a common language between graph signals, defined on irregular signal domains, and some of the most fundamental paradigms in DSP, such as spectral analysis of multichannel signals, system transfer function, digital filter design, parameter estimation, and optimal filter design. This is achieved through a physically meaningful and intuitive real-world example of geographically distributed multisensor temperature estimation. A similar spatial multisensor arrangement is already widely used in Signal Processing curricula to introduce minimum variance estimators and Kalman filters \cite{HM}, and by adopting this framework we facilitate a seamless integration of graph theory into the curriculum of existing DSP courses. By bridging the gap between standard approaches and graph signal processing, we also show that standard methods can be thought of as special cases of their graph counterparts, evaluated on linear graphs. It is hoped that our approach would not only help to demystify graph theoretic approaches in education and research but it would also empower practitioners to explore a whole host of otherwise prohibitive modern applications.
△ Less
Submitted 12 May, 2019; v1 submitted 26 March, 2019;
originally announced March 2019.
-
An Intuitive Derivation of the Coherence Index Relation in Compressive Sensing
Authors:
Ljubisa Stankovic,
Danilo Mandic,
Milos Dakovic,
Ilya Kisil
Abstract:
The existence and uniqueness conditions are a prerequisite for reliable reconstruction of sparse signals from reduced sets of measurements within the Compressive Sensing (CS) paradigm. However, despite their underpinning role for practical applications, existing uniqueness relations are either computationally prohibitive to implement (Restricted Isometry Property), or involve mathematical tools th…
▽ More
The existence and uniqueness conditions are a prerequisite for reliable reconstruction of sparse signals from reduced sets of measurements within the Compressive Sensing (CS) paradigm. However, despite their underpinning role for practical applications, existing uniqueness relations are either computationally prohibitive to implement (Restricted Isometry Property), or involve mathematical tools that are beyond the standard background of engineering graduates (Coherence Index). This can introduce conceptual and computational obstacles in the development of engineering intuition, the design of suboptimal practical solutions, or understanding of limitations. To this end, we introduce a simple but rigorous derivation of the coherence index condition, based on standard linear algebra, with the aim to empower signal processing practitioners with intuition in the design and ease in implementation of CS systems. Given that the coherence index is one of very few CS metrics that admits mathematically tractable and computationally feasible calculation, it is our hope that this work will help bridge the gap between the theory and applications of compressive sensing.
△ Less
Submitted 26 March, 2019;
originally announced March 2019.
-
Conditions for Unique Reconstruction of Sparse Signals Using Compressive Sensing Methods
Authors:
Ljubisa Stankovic,
Milos Dakovic,
Srdjan Stankovic,
Irena Orovic
Abstract:
A signal is sparse in one of its representation domain if the number of nonzero coefficients in that domain is much smaller than the total number of coefficients. Sparse signals can be reconstructed from a very reduced set of measurements/observations. The topic of this paper are conditions for the unique reconstruction of sparse signals from a reduced set of observations. After the basic definiti…
▽ More
A signal is sparse in one of its representation domain if the number of nonzero coefficients in that domain is much smaller than the total number of coefficients. Sparse signals can be reconstructed from a very reduced set of measurements/observations. The topic of this paper are conditions for the unique reconstruction of sparse signals from a reduced set of observations. After the basic definitions are introduced, the unique reconstruction conditions are reviewed using the spark, restricted isometry, and coherence of the measurement matrix. Uniqueness of the reconstruction of signals sparse in the discrete Fourier domain (DFT), as the most important signal transformation domain, is considered as well.
△ Less
Submitted 16 June, 2017;
originally announced June 2017.
-
Compressive Sensing of Sparse Signals in the Hermite Transform Basis: Analysis and Algorithm for Signal Reconstruction
Authors:
Miloš Brajovic,
Irena Orovic,
Milos Dakovic,
Srdjan Stankovic
Abstract:
An analysis of the influence of missing samples in signals exhibiting sparsity in the Hermite transform domain is provided. Based on the statistical properties derived for the Hermite coefficients of randomly undersampled signal, the probability of success in detection of signal components support is determined. Based on the probabilistic analysis, a threshold for the detection of signal component…
▽ More
An analysis of the influence of missing samples in signals exhibiting sparsity in the Hermite transform domain is provided. Based on the statistical properties derived for the Hermite coefficients of randomly undersampled signal, the probability of success in detection of signal components support is determined. Based on the probabilistic analysis, a threshold for the detection of signal components is provided. It is a crucial step in the definition of a simple non-iterative algorithm for compressive sensing signal reconstruction. The derived theoretical concepts are proved on several examples using different statistical tests.
△ Less
Submitted 14 November, 2015;
originally announced November 2015.
-
Hardware Architecture for Single Iteration Reconstruction Algorithm
Authors:
Andjela Draganic,
Irena Orovic,
Nedjeljko Lekic,
Milos Dakovic,
Srdjan Stankovic
Abstract:
A hardware architecture for the single iteration algorithm is proposed in this paper. Single iteration algorithm enables reconstruction of the full signal when small number of signal samples is available. The algorithm is based on the threshold calculation, and allows distinguishing between signal components and noise that appears as a consequence of missing samples. The proposed system for hardwa…
▽ More
A hardware architecture for the single iteration algorithm is proposed in this paper. Single iteration algorithm enables reconstruction of the full signal when small number of signal samples is available. The algorithm is based on the threshold calculation, and allows distinguishing between signal components and noise that appears as a consequence of missing samples. The proposed system for hardware realization is divided into three parts, each part with different functionality. The system is suitable for the FPGA realization. Realization of the blocks for which there are no standard components in FPGA, is discussed as well.
△ Less
Submitted 22 February, 2015;
originally announced February 2015.
-
Reconstruction of Randomly Sampled Sparse Signals Using an Adaptive Gradient Algorithm
Authors:
Ljubisa Stankovic,
Milos Dakovic
Abstract:
Sparse signals can be recovered from a reduced set of samples by using compressive sensing algorithms. In common methods the signal is recovered in the sparse domain. A method for the reconstruction of sparse signal which reconstructs the remaining missing samples/measurements is recently proposed. The available samples are fixed, while the missing samples are considered as minimization variables.…
▽ More
Sparse signals can be recovered from a reduced set of samples by using compressive sensing algorithms. In common methods the signal is recovered in the sparse domain. A method for the reconstruction of sparse signal which reconstructs the remaining missing samples/measurements is recently proposed. The available samples are fixed, while the missing samples are considered as minimization variables. Recovery of missing samples/measurements is done using an adaptive gradient-based algorithm in the time domain. A new criterion for the parameter adaptation in this algorithm, based on the gradient direction angles, is proposed. It improves the algorithm computational efficiency. A theorem for the uniqueness of the recovered signal for given set of missing samples (reconstruction variables) is presented. The case when available samples are a random subset of a uniformly or nonuniformly sampled signal is considered in this paper. A recalculation procedure is used to reconstruct the nonuniformly sampled signal. The methods are illustrated on statistical examples.
△ Less
Submitted 26 April, 2015; v1 submitted 1 December, 2014;
originally announced December 2014.
-
Adaptive Variable Step Algorithm for Missing Samples Recovery in Sparse Signals
Authors:
Ljubisa Stankovic,
Milos Dakovic,
Stefan Vujovic
Abstract:
Recovery of arbitrarily positioned samples that are missing in sparse signals recently attracted significant research interest. Sparse signals with heavily corrupted arbitrary positioned samples could be analyzed in the same way as compressive sensed signals by omitting the corrupted samples and considering them as unavailable during the recovery process. The reconstruction of missing samples is d…
▽ More
Recovery of arbitrarily positioned samples that are missing in sparse signals recently attracted significant research interest. Sparse signals with heavily corrupted arbitrary positioned samples could be analyzed in the same way as compressive sensed signals by omitting the corrupted samples and considering them as unavailable during the recovery process. The reconstruction of missing samples is done by using one of the well known reconstruction algorithms. In this paper we will propose a very simple and efficient adaptive variable step algorithm, applied directly to the concentration measures, without reformulating the reconstruction problem within the standard linear programming form. Direct application of the gradient approach to the nondifferentiable forms of measures lead us to introduce a variable step size algorithm. A criterion for changing adaptive algorithm parameters is presented. The results are illustrated on the examples with sparse signals, including approximately sparse signals and noisy sparse signals.
△ Less
Submitted 23 September, 2013;
originally announced September 2013.
