-
Graph Filters for Signal Processing and Machine Learning on Graphs
Authors:
Elvin Isufi,
Fernando Gama,
David I. Shuman,
Santiago Segarra
Abstract:
Filters are fundamental in extracting information from data. For time series and image data that reside on Euclidean domains, filters are the crux of many signal processing and machine learning techniques, including convolutional neural networks. Increasingly, modern data also reside on networks and other irregular domains whose structure is better captured by a graph. To process and learn from su…
▽ More
Filters are fundamental in extracting information from data. For time series and image data that reside on Euclidean domains, filters are the crux of many signal processing and machine learning techniques, including convolutional neural networks. Increasingly, modern data also reside on networks and other irregular domains whose structure is better captured by a graph. To process and learn from such data, graph filters account for the structure of the underlying data domain. In this article, we provide a comprehensive overview of graph filters, including the different filtering categories, design strategies for each type, and trade-offs between different types of graph filters. We discuss how to extend graph filters into filter banks and graph neural networks to enhance the representational power; that is, to model a broader variety of signal classes, data patterns, and relationships. We also showcase the fundamental role of graph filters in signal processing and machine learning applications. Our aim is that this article provides a unifying framework for both beginner and experienced researchers, as well as a common understanding that promotes collaborations at the intersections of signal processing, machine learning, and application domains.
△ Less
Submitted 19 February, 2024; v1 submitted 16 November, 2022;
originally announced November 2022.
-
Signal Processing on the Permutahedron: Tight Spectral Frames for Ranked Data Analysis
Authors:
Yilin Chen,
Jennifer DeJong,
Tom Halverson,
David I Shuman
Abstract:
Ranked data sets, where m judges/voters specify a preference ranking of n objects/candidates, are increasingly prevalent in contexts such as political elections, computer vision, recommender systems, and bioinformatics. The vote counts for each ranking can be viewed as an n! data vector lying on the permutahedron, which is a Cayley graph of the symmetric group with vertices labeled by permutations…
▽ More
Ranked data sets, where m judges/voters specify a preference ranking of n objects/candidates, are increasingly prevalent in contexts such as political elections, computer vision, recommender systems, and bioinformatics. The vote counts for each ranking can be viewed as an n! data vector lying on the permutahedron, which is a Cayley graph of the symmetric group with vertices labeled by permutations and an edge when two permutations differ by an adjacent transposition. Leveraging combinatorial representation theory and recent progress in signal processing on graphs, we investigate a novel, scalable transform method to interpret and exploit structure in ranked data. We represent data on the permutahedron using an overcomplete dictionary of atoms, each of which captures both smoothness information about the data (typically the focus of spectral graph decomposition methods in graph signal processing) and structural information about the data (typically the focus of symmetry decomposition methods from representation theory). These atoms have a more naturally interpretable structure than any known basis for signals on the permutahedron, and they form a Parseval frame, ensuring beneficial numerical properties such as energy preservation. We develop specialized algorithms and open software that take advantage of the symmetry and structure of the permutahedron to improve the scalability of the proposed method, making it more applicable to the high-dimensional ranked data found in applications.
△ Less
Submitted 15 August, 2021; v1 submitted 6 March, 2021;
originally announced March 2021.
-
Localized Spectral Graph Filter Frames: A Unifying Framework, Survey of Design Considerations, and Numerical Comparison (Extended Cut)
Authors:
David I Shuman
Abstract:
Representing data residing on a graph as a linear combination of building block signals can enable efficient and insightful visual or statistical analysis of the data, and such representations prove useful as regularizers in signal processing and machine learning tasks. Designing collections of building block signals -- or more formally, dictionaries of atoms -- that specifically account for the u…
▽ More
Representing data residing on a graph as a linear combination of building block signals can enable efficient and insightful visual or statistical analysis of the data, and such representations prove useful as regularizers in signal processing and machine learning tasks. Designing collections of building block signals -- or more formally, dictionaries of atoms -- that specifically account for the underlying graph structure as well as any available representative training signals has been an active area of research over the last decade. In this article, we survey a particular class of dictionaries called localized spectral graph filter frames, whose atoms are created by localizing spectral patterns to different regions of the graph. After showing how this class encompasses a variety of approaches from spectral graph wavelets to graph filter banks, we focus on the two main questions of how to design the spectral filters and how to select the center vertices to which the patterns are localized. Throughout, we emphasize computationally efficient methods that ensure the resulting transforms and their inverses can be applied to data residing on large, sparse graphs. We demonstrate how this class of transform methods can be used in signal processing tasks such as denoising and non-linear approximation, and provide code for readers to experiment with these methods in new application domains.
△ Less
Submitted 4 August, 2020; v1 submitted 19 June, 2020;
originally announced June 2020.
-
Scalable $M$-Channel Critically Sampled Filter Banks for Graph Signals
Authors:
Shuni Li,
Yan **,
David I Shuman
Abstract:
We investigate a scalable $M$-channel critically sampled filter bank for graph signals, where each of the $M$ filters is supported on a different subband of the graph Laplacian spectrum. For analysis, the graph signal is filtered on each subband and downsampled on a corresponding set of vertices. However, the classical synthesis filters are replaced with interpolation operators. For small graphs,…
▽ More
We investigate a scalable $M$-channel critically sampled filter bank for graph signals, where each of the $M$ filters is supported on a different subband of the graph Laplacian spectrum. For analysis, the graph signal is filtered on each subband and downsampled on a corresponding set of vertices. However, the classical synthesis filters are replaced with interpolation operators. For small graphs, we use a full eigendecomposition of the graph Laplacian to partition the graph vertices such that the $m^{th}$ set comprises a uniqueness set for signals supported on the $m^{th}$ subband. The resulting transform is critically sampled, the dictionary atoms are orthogonal to those supported on different bands, and graph signals are perfectly reconstructable from their analysis coefficients. We also investigate fast versions of the proposed transform that scale efficiently for large, sparse graphs. Issues that arise in this context include designing the filter bank to be more amenable to polynomial approximation, estimating the number of samples required for each band, performing non-uniform random sampling for the filtered signals on each band, and using efficient reconstruction methods. We empirically explore the joint vertex-frequency localization of the dictionary atoms, the sparsity of the analysis coefficients for different classes of signals, the reconstruction error resulting from the numerical approximations, and the ability of the proposed transform to compress piecewise-smooth graph signals. The proposed filter bank also yields a fast, approximate graph Fourier transform with a coarse resolution in the spectral domain.
△ Less
Submitted 22 January, 2019; v1 submitted 10 August, 2016;
originally announced August 2016.
-
GSPBOX: A toolbox for signal processing on graphs
Authors:
Nathanaël Perraudin,
Johan Paratte,
David Shuman,
Lionel Martin,
Vassilis Kalofolias,
Pierre Vandergheynst,
David K. Hammond
Abstract:
This document introduces the Graph Signal Processing Toolbox (GSPBox) a framework that can be used to tackle graph related problems with a signal processing approach. It explains the structure and the organization of this software. It also contains a general description of the important modules.
This document introduces the Graph Signal Processing Toolbox (GSPBox) a framework that can be used to tackle graph related problems with a signal processing approach. It explains the structure and the organization of this software. It also contains a general description of the important modules.
△ Less
Submitted 15 March, 2016; v1 submitted 25 August, 2014;
originally announced August 2014.
-
UNLocBoX: A MATLAB convex optimization toolbox for proximal-splitting methods
Authors:
Nathanael Perraudin,
Vassilis Kalofolias,
David Shuman,
Pierre Vandergheynst
Abstract:
Convex optimization is an essential tool for machine learning, as many of its problems can be formulated as minimization problems of specific objective functions. While there is a large variety of algorithms available to solve convex problems, we can argue that it becomes more and more important to focus on efficient, scalable methods that can deal with big data. When the objective function can be…
▽ More
Convex optimization is an essential tool for machine learning, as many of its problems can be formulated as minimization problems of specific objective functions. While there is a large variety of algorithms available to solve convex problems, we can argue that it becomes more and more important to focus on efficient, scalable methods that can deal with big data. When the objective function can be written as a sum of "simple" terms, proximal splitting methods are a good choice. UNLocBoX is a MATLAB library that implements many of these methods, designed to solve convex optimization problems of the form $\min_{x \in \mathbb{R}^N} \sum_{n=1}^K f_n(x).$ It contains the most recent solvers such as FISTA, Douglas-Rachford, SDMM as well a primal dual techniques such as Chambolle-Pock and forward-backward-forward. It also includes an extensive list of common proximal operators that can be combined, allowing for a quick implementation of a large variety of convex problems.
△ Less
Submitted 27 December, 2016; v1 submitted 4 February, 2014;
originally announced February 2014.
-
Learning parametric dictionaries for graph signals
Authors:
Dorina Thanou,
David I Shuman,
Pascal Frossard
Abstract:
In sparse signal representation, the choice of a dictionary often involves a tradeoff between two desirable properties -- the ability to adapt to specific signal data and a fast implementation of the dictionary. To sparsely represent signals residing on weighted graphs, an additional design challenge is to incorporate the intrinsic geometric structure of the irregular data domain into the atoms of…
▽ More
In sparse signal representation, the choice of a dictionary often involves a tradeoff between two desirable properties -- the ability to adapt to specific signal data and a fast implementation of the dictionary. To sparsely represent signals residing on weighted graphs, an additional design challenge is to incorporate the intrinsic geometric structure of the irregular data domain into the atoms of the dictionary. In this work, we propose a parametric dictionary learning algorithm to design data-adapted, structured dictionaries that sparsely represent graph signals. In particular, we model graph signals as combinations of overlap** local patterns. We impose the constraint that each dictionary is a concatenation of subdictionaries, with each subdictionary being a polynomial of the graph Laplacian matrix, representing a single pattern translated to different areas of the graph. The learning algorithm adapts the patterns to a training set of graph signals. Experimental results on both synthetic and real datasets demonstrate that the dictionaries learned by the proposed algorithm are competitive with and often better than unstructured dictionaries learned by state-of-the-art numerical learning algorithms in terms of sparse approximation of graph signals. In contrast to the unstructured dictionaries, however, the dictionaries learned by the proposed algorithm feature localized atoms and can be implemented in a computationally efficient manner in signal processing tasks such as compression, denoising, and classification.
△ Less
Submitted 5 January, 2014;
originally announced January 2014.
-
Spectrum-Adapted Tight Graph Wavelet and Vertex-Frequency Frames
Authors:
David I Shuman,
Christoph Wiesmeyr,
Nicki Holighaus,
Pierre Vandergheynst
Abstract:
We consider the problem of designing spectral graph filters for the construction of dictionaries of atoms that can be used to efficiently represent signals residing on weighted graphs. While the filters used in previous spectral graph wavelet constructions are only adapted to the length of the spectrum, the filters proposed in this paper are adapted to the distribution of graph Laplacian eigenvalu…
▽ More
We consider the problem of designing spectral graph filters for the construction of dictionaries of atoms that can be used to efficiently represent signals residing on weighted graphs. While the filters used in previous spectral graph wavelet constructions are only adapted to the length of the spectrum, the filters proposed in this paper are adapted to the distribution of graph Laplacian eigenvalues, and therefore lead to atoms with better discriminatory power. Our approach is to first characterize a family of systems of uniformly translated kernels in the graph spectral domain that give rise to tight frames of atoms generated via generalized translation on the graph. We then warp the uniform translates with a function that approximates the cumulative spectral density function of the graph Laplacian eigenvalues. We use this approach to construct computationally efficient, spectrum-adapted, tight vertex-frequency and graph wavelet frames. We give numerous examples of the resulting spectrum-adapted graph filters, and also present an illustrative example of vertex-frequency analysis using the proposed construction.
△ Less
Submitted 4 November, 2013;
originally announced November 2013.
-
A Multiscale Pyramid Transform for Graph Signals
Authors:
David I Shuman,
Mohammad Javad Faraji,
Pierre Vandergheynst
Abstract:
Multiscale transforms designed to process analog and discrete-time signals and images cannot be directly applied to analyze high-dimensional data residing on the vertices of a weighted graph, as they do not capture the intrinsic geometric structure of the underlying graph data domain. In this paper, we adapt the Laplacian pyramid transform for signals on Euclidean domains so that it can be used to…
▽ More
Multiscale transforms designed to process analog and discrete-time signals and images cannot be directly applied to analyze high-dimensional data residing on the vertices of a weighted graph, as they do not capture the intrinsic geometric structure of the underlying graph data domain. In this paper, we adapt the Laplacian pyramid transform for signals on Euclidean domains so that it can be used to analyze high-dimensional data residing on the vertices of a weighted graph. Our approach is to study existing methods and develop new methods for the four fundamental operations of graph downsampling, graph reduction, and filtering and interpolation of signals on graphs. Equipped with appropriate notions of these operations, we leverage the basic multiscale constructs and intuitions from classical signal processing to generate a transform that yields both a multiresolution of graphs and an associated multiresolution of a graph signal on the underlying sequence of graphs.
△ Less
Submitted 14 March, 2016; v1 submitted 22 August, 2013;
originally announced August 2013.
-
Vertex-Frequency Analysis on Graphs
Authors:
David I Shuman,
Benjamin Ricaud,
Pierre Vandergheynst
Abstract:
One of the key challenges in the area of signal processing on graphs is to design dictionaries and transform methods to identify and exploit structure in signals on weighted graphs. To do so, we need to account for the intrinsic geometric structure of the underlying graph data domain. In this paper, we generalize one of the most important signal processing tools - windowed Fourier analysis - to th…
▽ More
One of the key challenges in the area of signal processing on graphs is to design dictionaries and transform methods to identify and exploit structure in signals on weighted graphs. To do so, we need to account for the intrinsic geometric structure of the underlying graph data domain. In this paper, we generalize one of the most important signal processing tools - windowed Fourier analysis - to the graph setting. Our approach is to first define generalized convolution, translation, and modulation operators for signals on graphs, and explore related properties such as the localization of translated and modulated graph kernels. We then use these operators to define a windowed graph Fourier transform, enabling vertex-frequency analysis. When we apply this transform to a signal with frequency components that vary along a path graph, the resulting spectrogram matches our intuition from classical discrete-time signal processing. Yet, our construction is fully generalized and can be applied to analyze signals on any undirected, connected, weighted graph.
△ Less
Submitted 22 July, 2013;
originally announced July 2013.
-
The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
Authors:
David I Shuman,
Sunil K. Narang,
Pascal Frossard,
Antonio Ortega,
Pierre Vandergheynst
Abstract:
In applications such as social, energy, transportation, sensor, and neuronal networks, high-dimensional data naturally reside on the vertices of weighted graphs. The emerging field of signal processing on graphs merges algebraic and spectral graph theoretic concepts with computational harmonic analysis to process such signals on graphs. In this tutorial overview, we outline the main challenges of…
▽ More
In applications such as social, energy, transportation, sensor, and neuronal networks, high-dimensional data naturally reside on the vertices of weighted graphs. The emerging field of signal processing on graphs merges algebraic and spectral graph theoretic concepts with computational harmonic analysis to process such signals on graphs. In this tutorial overview, we outline the main challenges of the area, discuss different ways to define graph spectral domains, which are the analogues to the classical frequency domain, and highlight the importance of incorporating the irregular structures of graph data domains when processing signals on graphs. We then review methods to generalize fundamental operations such as filtering, translation, modulation, dilation, and downsampling to the graph setting, and survey the localized, multiscale transforms that have been proposed to efficiently extract information from high-dimensional data on graphs. We conclude with a brief discussion of open issues and possible extensions.
△ Less
Submitted 10 March, 2013; v1 submitted 31 October, 2012;
originally announced November 2012.
-
Distributed Signal Processing via Chebyshev Polynomial Approximation
Authors:
David I Shuman,
Pierre Vandergheynst,
Daniel Kressner,
Pascal Frossard
Abstract:
Unions of graph multiplier operators are an important class of linear operators for processing signals defined on graphs. We present a novel method to efficiently distribute the application of these operators. The proposed method features approximations of the graph multipliers by shifted Chebyshev polynomials, whose recurrence relations make them readily amenable to distributed computation. We de…
▽ More
Unions of graph multiplier operators are an important class of linear operators for processing signals defined on graphs. We present a novel method to efficiently distribute the application of these operators. The proposed method features approximations of the graph multipliers by shifted Chebyshev polynomials, whose recurrence relations make them readily amenable to distributed computation. We demonstrate how the proposed method can be applied to distributed processing tasks such as smoothing, denoising, inverse filtering, and semi-supervised classification, and show that the communication requirements of the method scale gracefully with the size of the network.
△ Less
Submitted 29 July, 2017; v1 submitted 22 November, 2011;
originally announced November 2011.
-
Chebyshev Polynomial Approximation for Distributed Signal Processing
Authors:
David I Shuman,
Pierre Vandergheynst,
Pascal Frossard
Abstract:
Unions of graph Fourier multipliers are an important class of linear operators for processing signals defined on graphs. We present a novel method to efficiently distribute the application of these operators to the high-dimensional signals collected by sensor networks. The proposed method features approximations of the graph Fourier multipliers by shifted Chebyshev polynomials, whose recurrence re…
▽ More
Unions of graph Fourier multipliers are an important class of linear operators for processing signals defined on graphs. We present a novel method to efficiently distribute the application of these operators to the high-dimensional signals collected by sensor networks. The proposed method features approximations of the graph Fourier multipliers by shifted Chebyshev polynomials, whose recurrence relations make them readily amenable to distributed computation. We demonstrate how the proposed method can be used in a distributed denoising task, and show that the communication requirements of the method scale gracefully with the size of the network.
△ Less
Submitted 27 May, 2011; v1 submitted 10 May, 2011;
originally announced May 2011.
-
Energy-Efficient Transmission Scheduling with Strict Underflow Constraints
Authors:
David I Shuman,
Mingyan Liu,
Owen Q. Wu
Abstract:
We consider a single source transmitting data to one or more receivers/users over a shared wireless channel. Due to random fading, the wireless channel conditions vary with time and from user to user. Each user has a buffer to store received packets before they are drained. At each time step, the source determines how much power to use for transmission to each user. The source's objective is to…
▽ More
We consider a single source transmitting data to one or more receivers/users over a shared wireless channel. Due to random fading, the wireless channel conditions vary with time and from user to user. Each user has a buffer to store received packets before they are drained. At each time step, the source determines how much power to use for transmission to each user. The source's objective is to allocate power in a manner that minimizes an expected cost measure, while satisfying strict buffer underflow constraints and a total power constraint in each slot. The expected cost measure is composed of costs associated with power consumption from transmission and packet holding costs. The primary application motivating this problem is wireless media streaming. For this application, the buffer underflow constraints prevent the user buffers from emptying, so as to maintain playout quality. In the case of a single user with linear power-rate curves, we show that a modified base-stock policy is optimal under the finite horizon, infinite horizon discounted, and infinite horizon average expected cost criteria. For a single user with piecewise-linear convex power-rate curves, we show that a finite generalized base-stock policy is optimal under all three expected cost criteria. We also present the sequences of critical numbers that complete the characterization of the optimal control laws in each of these cases when some additional technical conditions are satisfied. We then analyze the structure of the optimal policy for the case of two users. We conclude with a discussion of methods to identify implementable near-optimal policies for the most general case of M users.
△ Less
Submitted 15 February, 2010; v1 submitted 12 August, 2009;
originally announced August 2009.
