Showing 1–11 of 11 results for author: Knyazev, A

Preconditioned Spectral Clustering for Stochastic Block Partition Streaming Graph Challenge

Authors: David Zhuzhunashvili, Andrew Knyazev

Abstract: Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) is demonstrated to efficiently solve eigenvalue problems for graph Laplacians that appear in spectral clustering. For static graph partitioning, 10-20 iterations of LOBPCG without preconditioning result in ~10x error reduction, enough to achieve 100% correctness for all Challenge datasets with known truth partitions, e.g., for graphs… ▽ More Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) is demonstrated to efficiently solve eigenvalue problems for graph Laplacians that appear in spectral clustering. For static graph partitioning, 10-20 iterations of LOBPCG without preconditioning result in ~10x error reduction, enough to achieve 100% correctness for all Challenge datasets with known truth partitions, e.g., for graphs with 5K/.1M (50K/1M) Vertices/Edges in 2 (7) seconds, compared to over 5,000 (30,000) seconds needed by the baseline Python code. Our Python code 100% correctly determines 98 (160) clusters from the Challenge static graphs with 0.5M (2M) vertices in 270 (1,700) seconds using 10GB (50GB) of memory. Our single-precision MATLAB code calculates the same clusters at half time and memory. For streaming graph partitioning, LOBPCG is initiated with approximate eigenvectors of the graph Laplacian already computed for the previous graph, in many cases reducing 2-3 times the number of required LOBPCG iterations, compared to the static case. Our spectral clustering is generic, i.e. assuming nothing specific of the block model or streaming, used to generate the graphs for the Challenge, in contrast to the base code. Nevertheless, in 10-stage streaming comparison with the base code for the 5K graph, the quality of our clusters is similar or better starting at stage 4 (7) for emerging edging (snowballing) streaming, while the computations are over 100-1000 faster. △ Less

Submitted 21 August, 2017; originally announced August 2017.

Comments: 6 pages. To appear in Proceedings of the 2017 IEEE High Performance Extreme Computing Conference. Student Innovation Award Streaming Graph Challenge: Stochastic Block Partition, see http://graphchallenge.mit.edu/champions

MSC Class: 05C50; 05C70; 15A18; 58C40; 65F15; 65N25; 62H30; 91C20 ACM Class: H.3.3; I.5.3

Journal ref: 2017 IEEE High Performance Extreme Computing Conference (HPEC), Waltham, MA, USA, 2017, pp. 1-6

Signal reconstruction via operator guiding

Authors: Andrew Knyazev, Alexander Malyshev

Abstract: Signal reconstruction from a sample using an orthogonal projector onto a guiding subspace is theoretically well justified, but may be difficult to practically implement. We propose more general guiding operators, which increase signal components in the guiding subspace relative to those in a complementary subspace, e.g., iterative low-pass edge-preserving filters for super-resolution of images. Tw… ▽ More Signal reconstruction from a sample using an orthogonal projector onto a guiding subspace is theoretically well justified, but may be difficult to practically implement. We propose more general guiding operators, which increase signal components in the guiding subspace relative to those in a complementary subspace, e.g., iterative low-pass edge-preserving filters for super-resolution of images. Two examples of super-resolution illustrate our technology: a no-flash RGB photo guided using a high resolution flash RGB photo, and a depth image guided using a high resolution RGB photo. △ Less

Submitted 9 May, 2017; originally announced May 2017.

Comments: 5 pages, 8 figures. To appear in Proceedings of SampTA 2017: Sampling Theory and Applications, 12th International Conference, July 3-7, 2017, Tallinn, Estonia

Report number: MERL TR2017-087 MSC Class: 65F30 ACM Class: I.4.3; G.1.3

Journal ref: IEEE Xplore: 2017 International Conference on Sampling Theory and Applications (SampTA), Tallin, Estonia, 2017, pp. 630-634

Guided Signal Reconstruction Theory

Authors: Andrew Knyazev, Akshay Gadde, Hassan Mansour, Dong Tian

Abstract: An axiomatic approach to signal reconstruction is formulated, involving a sample consistent set and a guiding set, describing desired reconstructions. New frame-less reconstruction methods are proposed, based on a novel concept of a reconstruction set, defined as a shortest pathway between the sample consistent set and the guiding set. Existence and uniqueness of the reconstruction set are investi… ▽ More An axiomatic approach to signal reconstruction is formulated, involving a sample consistent set and a guiding set, describing desired reconstructions. New frame-less reconstruction methods are proposed, based on a novel concept of a reconstruction set, defined as a shortest pathway between the sample consistent set and the guiding set. Existence and uniqueness of the reconstruction set are investigated in a Hilbert space, where the guiding set is a closed subspace and the sample consistent set is a closed plane, formed by a sampling subspace. Connections to earlier known consistent, generalized, and regularized reconstructions are clarified. New stability and reconstruction error bounds are derived, using the largest nontrivial angle between the sampling and guiding subspaces. Conjugate gradient iterative reconstruction algorithms are proposed and illustrated numerically for image magnification. △ Less

Submitted 2 February, 2017; originally announced February 2017.

Comments: 20 pages, 11 figures

Report number: MERL TR2017-020 MSC Class: 46C07; 46N99; 94A08; 94A12; 94A20; 94A24 ACM Class: E.4; H.1.1; H.5.5; I.4.2; I.4.8

On spectral partitioning of signed graphs

Authors: Andrew V. Knyazev

Abstract: We argue that the standard graph Laplacian is preferable for spectral partitioning of signed graphs compared to the signed Laplacian. Simple examples demonstrate that partitioning based on signs of components of the leading eigenvectors of the signed Laplacian may be meaningless, in contrast to partitioning based on the Fiedler vector of the standard graph Laplacian for signed graphs. We observe t… ▽ More We argue that the standard graph Laplacian is preferable for spectral partitioning of signed graphs compared to the signed Laplacian. Simple examples demonstrate that partitioning based on signs of components of the leading eigenvectors of the signed Laplacian may be meaningless, in contrast to partitioning based on the Fiedler vector of the standard graph Laplacian for signed graphs. We observe that negative eigenvalues are beneficial for spectral partitioning of signed graphs, making the Fiedler vector easier to compute. △ Less

Submitted 29 March, 2018; v1 submitted 5 January, 2017; originally announced January 2017.

Comments: 12 pages, 10 figures. Rev 2 to appear in proceedings of the SIAM Workshop on Combinatorial Scientific Computing 2018 (CSC18)

Report number: Rev. 1 MERL TR2017-001 MSC Class: 05C50; 05C70; 15A18; 58C40; 65F15; 65N25; 62H30; 91C20 ACM Class: H.3.3; I.5.3

Accelerated graph-based nonlinear denoising filters

Authors: Andrew Knyazev, Alexander Malyshev

Abstract: Denoising filters, such as bilateral, guided, and total variation filters, applied to images on general graphs may require repeated application if noise is not small enough. We formulate two acceleration techniques of the resulted iterations: conjugate gradient method and Nesterov's acceleration. We numerically show efficiency of the accelerated nonlinear filters for image denoising and demonstrat… ▽ More Denoising filters, such as bilateral, guided, and total variation filters, applied to images on general graphs may require repeated application if noise is not small enough. We formulate two acceleration techniques of the resulted iterations: conjugate gradient method and Nesterov's acceleration. We numerically show efficiency of the accelerated nonlinear filters for image denoising and demonstrate 2-12 times speed-up, i.e., the acceleration techniques reduce the number of iterations required to reach a given peak signal-to-noise ratio (PSNR) by the above indicated factor of 2-12. △ Less

Submitted 13 April, 2016; v1 submitted 1 December, 2015; originally announced December 2015.

Comments: 10 pages, 6 figures, to appear in Procedia Computer Science, vol.80, 2016, International Conference on Computational Science, San Diego, CA, USA, June 6-8, 2016

Report number: MERL TR2016-068 MSC Class: 65F30 ACM Class: I.4.3; G.1.3

Journal ref: Procedia Computer Science Volume 80, 2016, Pages 607-616, International Conference on Computational Science 2016, ICCS 2016, 6-8 June 2016, San Diego, California, USA

Edge-enhancing Filters with Negative Weights

Authors: Andrew Knyazev

Abstract: In [DOI:10.1109/ICMEW.2014.6890711], a graph-based denoising is performed by projecting the noisy image to a lower dimensional Krylov subspace of the graph Laplacian, constructed using nonnegative weights determined by distances between image data corresponding to image pixels. We~extend the construction of the graph Laplacian to the case, where some graph weights can be negative. Removing the pos… ▽ More In [DOI:10.1109/ICMEW.2014.6890711], a graph-based denoising is performed by projecting the noisy image to a lower dimensional Krylov subspace of the graph Laplacian, constructed using nonnegative weights determined by distances between image data corresponding to image pixels. We~extend the construction of the graph Laplacian to the case, where some graph weights can be negative. Removing the positivity constraint provides a more accurate inference of a graph model behind the data, and thus can improve quality of filters for graph-based signal processing, e.g., denoising, compared to the standard construction, without affecting the costs. △ Less

Submitted 8 September, 2015; originally announced September 2015.

Comments: 5 pages; 6 figures. Accepted to IEEE GlobalSIP 2015 conference

Report number: MERL-TR2015-142 MSC Class: 68U10; 05C85 ACM Class: I.4.3; I.4.6; I.5.4

Journal ref: 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, FL, 14-16 Dec.2015, pp. 260 - 264

Accelerated graph-based spectral polynomial filters

Authors: Andrew Knyazev, Alexander Malyshev

Abstract: Graph-based spectral denoising is a low-pass filtering using the eigendecomposition of the graph Laplacian matrix of a noisy signal. Polynomial filtering avoids costly computation of the eigendecomposition by projections onto suitable Krylov subspaces. Polynomial filters can be based, e.g., on the bilateral and guided filters. We propose constructing accelerated polynomial filters by running flexi… ▽ More Graph-based spectral denoising is a low-pass filtering using the eigendecomposition of the graph Laplacian matrix of a noisy signal. Polynomial filtering avoids costly computation of the eigendecomposition by projections onto suitable Krylov subspaces. Polynomial filters can be based, e.g., on the bilateral and guided filters. We propose constructing accelerated polynomial filters by running flexible Krylov subspace based linear and eigenvalue solvers such as the Block Locally Optimal Preconditioned Conjugate Gradient (LOBPCG) method. △ Less

Submitted 8 September, 2015; originally announced September 2015.

Comments: 6 pages, 6 figures. Accepted to the 2015 IEEE International Workshop on Machine Learning for Signal Processing

Report number: MERL-TR2015-106

Journal ref: Machine Learning for Signal Processing (MLSP), 2015 IEEE 25th International Workshop on , pp.1-6, 17-20 Sept. 2015

Guided Signal Reconstruction with Application to Image Magnification

Authors: Akshay Gadde, Andrew Knyazev, Dong Tian, Hassan Mansour

Abstract: We study the problem of reconstructing a signal from its projection on a subspace. The proposed signal reconstruction algorithms utilize a guiding subspace that represents desired properties of reconstructed signals. We show that optimal reconstructed signals belong to a convex bounded set, called the "reconstruction" set. We also develop iterative algorithms, based on conjugate gradient methods,… ▽ More We study the problem of reconstructing a signal from its projection on a subspace. The proposed signal reconstruction algorithms utilize a guiding subspace that represents desired properties of reconstructed signals. We show that optimal reconstructed signals belong to a convex bounded set, called the "reconstruction" set. We also develop iterative algorithms, based on conjugate gradient methods, to approximate optimal reconstructions with low memory and computational costs. The effectiveness of the proposed approach is demonstrated for image magnification, where the reconstructed image quality is shown to exceed that of consistent and generalized reconstruction schemes. △ Less

Submitted 8 September, 2015; originally announced September 2015.

Comments: 5 pages, 6 figures; Accepted to IEEE GlobalSIP 2015

Report number: MERL-TR2015-141

Journal ref: 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, FL, 14-16 Dec.2015, pp. 938 - 942

Chebyshev and Conjugate Gradient Filters for Graph Image Denoising

Authors: Dong Tian, Hassan Mansour, Andrew Knyazev, Anthony Vetro

Abstract: In 3D image/video acquisition, different views are often captured with varying noise levels across the views. In this paper, we propose a graph-based image enhancement technique that uses a higher quality view to enhance a degraded view. A depth map is utilized as auxiliary information to match the perspectives of the two views. Our method performs graph-based filtering of the noisy image by direc… ▽ More In 3D image/video acquisition, different views are often captured with varying noise levels across the views. In this paper, we propose a graph-based image enhancement technique that uses a higher quality view to enhance a degraded view. A depth map is utilized as auxiliary information to match the perspectives of the two views. Our method performs graph-based filtering of the noisy image by directly computing a projection of the image to be filtered onto a lower dimensional Krylov subspace of the graph Laplacian. We discuss two graph spectral denoising methods: first using Chebyshev polynomials, and second using iterations of the conjugate gradient algorithm. Our framework generalizes previously known polynomial graph filters, and we demonstrate through numerical simulations that our proposed technique produces subjectively cleaner images with about 1-3 dB improvement in PSNR over existing polynomial graph filters. △ Less

Submitted 4 September, 2015; originally announced September 2015.

Comments: 6 pages, 6 figures, accepted to 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW)

Report number: MERL TR2014-062

Journal ref: Multimedia and Expo Workshops (ICMEW), 2014 IEEE International Conference on, vol., no., pp.1-6, 14-18 July 2014

Conjugate Gradient Acceleration of Non-Linear Smoothing Filters

Authors: Andrew Knyazev, Alexander Malyshev

Abstract: The most efficient signal edge-preserving smoothing filters, e.g., for denoising, are non-linear. Thus, their acceleration is challenging and is often performed in practice by tuning filter parameters, such as by increasing the width of the local smoothing neighborhood, resulting in more aggressive smoothing of a single sweep at the cost of increased edge blurring. We propose an alternative techno… ▽ More The most efficient signal edge-preserving smoothing filters, e.g., for denoising, are non-linear. Thus, their acceleration is challenging and is often performed in practice by tuning filter parameters, such as by increasing the width of the local smoothing neighborhood, resulting in more aggressive smoothing of a single sweep at the cost of increased edge blurring. We propose an alternative technology, accelerating the original filters without tuning, by running them through a special conjugate gradient method, not affecting their quality. The filter non-linearity is dealt with by careful freezing and restarting. Our initial numerical experiments on toy one-dimensional signals demonstrate 20x acceleration of the classical bilateral filter and 3-5x acceleration of the recently developed guided filter. △ Less

Submitted 4 September, 2015; originally announced September 2015.

Comments: 5 pages, 5 figures, IEEE Conference GlobalSIP 2015

Report number: MERL TR2015-143

Journal ref: 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, FL, 2015, pp. 245-249

Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc

Authors: A. V. Knyazev, M. E. Argentati, I. Lashuk, E. E. Ovtchinnikov

Abstract: We describe our software package Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) publicly released recently. BLOPEX is available as a stand-alone serial library, as an external package to PETSc (``Portable, Extensible Toolkit for Scientific Computation'', a general purpose suite of tools for the scalable solution of partial differential equations and related problems developed b… ▽ More We describe our software package Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) publicly released recently. BLOPEX is available as a stand-alone serial library, as an external package to PETSc (``Portable, Extensible Toolkit for Scientific Computation'', a general purpose suite of tools for the scalable solution of partial differential equations and related problems developed by Argonne National Laboratory), and is also built into {\it hypre} (``High Performance Preconditioners'', scalable linear solvers package developed by Lawrence Livermore National Laboratory). The present BLOPEX release includes only one solver--the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) method for symmetric eigenvalue problems. {\it hypre} provides users with advanced high-quality parallel preconditioners for linear systems, in particular, with domain decomposition and multigrid preconditioners. With BLOPEX, the same preconditioners can now be efficiently used for symmetric eigenvalue problems. PETSc facilitates the integration of independently developed application modules with strict attention to component interoperability, and makes BLOPEX extremely easy to compile and use with preconditioners that are available via PETSc. We present the LOBPCG algorithm in BLOPEX for {\it hypre} and PETSc. We demonstrate numerically the scalability of BLOPEX by testing it on a number of distributed and shared memory parallel systems, including a Beowulf system, SUN Fire 880, an AMD dual-core Opteron workstation, and IBM BlueGene/L supercomputer, using PETSc domain decomposition and {\it hypre} multigrid preconditioning. We test BLOPEX on a model problem, the standard 7-point finite-difference approximation of the 3-D Laplacian, with the problem size in the range $10^5-10^8$. △ Less

Submitted 17 May, 2007; originally announced May 2007.

Comments: Submitted to SIAM Journal on Scientific Computing

Report number: UCDHSC-CCM-251 ACM Class: G.4; G.1.3; G.1.8

Journal ref: SIAM Journal on Scientific Computing (SISC). 25(5): 2224-2239, 2007