[Paper Review] Nearly-Linear Time Spectral Graph Reduction for Scalable Graph Partitioning and Data Visualization

Zhiqiang Zhao, Yongyu Wang+1 more|arXiv (Cornell University)|Dec 21, 2018

Image and Video Quality Assessment32 references10 citations

TL;DR

This paper proposes a nearly-linear time spectral graph reduction framework that preserves key spectral properties of large undirected graphs through spectrum-preserving node aggregation and iterative spectral sparsification with constrained stochastic gradient descent. The method reduces graphs like coPapersCiteseer (0.43M nodes, 16M edges) to 13K nodes and 17K edges in ~16 seconds, enabling up to 1100X speedup in spectral partitioning and 60X speedup in t-SNE visualization while maintaining high solution quality.

ABSTRACT

This paper proposes a scalable algorithmic framework for spectral reduction of large undirected graphs. The proposed method allows computing much smaller graphs while preserving the key spectral (structural) properties of the original graph. Our framework is built upon the following two key components: a spectrum-preserving node aggregation (reduction) scheme, as well as a spectral graph sparsification framework with iterative edge weight scaling. We show that the resulting spectrally-reduced graphs can robustly preserve the first few nontrivial eigenvalues and eigenvectors of the original graph Laplacian. In addition, the spectral graph reduction method has been leveraged to develop much faster algorithms for multilevel spectral graph partitioning as well as t-distributed Stochastic Neighbor Embedding (t-SNE) of large data sets. We conducted extensive experiments using a variety of large graphs and data sets, and obtained very promising results. For instance, we are able to reduce the "coPapersCiteseer" graph with 0.43 million nodes and 16 million edges to a much smaller graph with only 13K (32X fewer) nodes and 17K (950X fewer) edges in about 16 seconds; the spectrally-reduced graphs also allow us to achieve up to 1100X speedup for spectral graph partitioning and up to 60X speedup for t-SNE visualization of large data sets.

Motivation & Objective

• To develop a scalable, nearly-linear time algorithm for spectral reduction of large undirected graphs that preserves the first few nontrivial eigenvalues and eigenvectors of the graph Laplacian.
• To enable faster numerical and graph algorithms by drastically reducing graph size (nodes and edges) while maintaining spectral fidelity.
• To integrate spectral reduction into multilevel frameworks for accelerating spectral graph partitioning and t-distributed Stochastic Neighbor Embedding (t-SNE).
• To achieve high performance and solution quality through a novel constrained stochastic gradient descent optimization approach for iterative edge weight scaling.
• To demonstrate the effectiveness of spectrally-reduced graphs on real-world large-scale graphs and data sets, showing significant speedups without loss of accuracy.

Proposed method

• A spectrum-preserving node aggregation scheme is used to coarsen the graph by merging similar nodes while preserving spectral structure.
• A spectral graph sparsification framework with iterative edge weight scaling is applied to maintain sparsity and spectral fidelity in the reduced graph.
• A novel constrained stochastic gradient descent (SGD) optimization method is employed to refine edge weights during sparsification, ensuring robust preservation of Laplacian eigenvectors.
• The method leverages similarity-aware spectral sparsification and graph-theoretic algebraic multigrid (AMG) Laplacian solvers for nearly-linear time complexity.
• A refinement procedure is introduced to improve the quality of eigenvectors and other solutions computed on the spectrally-reduced graphs.
• The framework is integrated into multilevel algorithms for spectral graph partitioning and t-SNE to accelerate computation on large data sets.

Experimental results

Research questions

• RQ1 Can a nearly-linear time algorithm preserve the first few nontrivial eigenvalues and eigenvectors of the original graph Laplacian during graph reduction?
• RQ2 To what extent can spectral graph reduction reduce graph size (nodes and edges) while maintaining spectral fidelity for downstream tasks?
• RQ3 How effective is the proposed framework in accelerating multilevel spectral graph partitioning and t-SNE visualization on large-scale graphs and data sets?
• RQ4 What is the impact of including higher-order eigenvectors (e.g., 10th) on the correlation with t-SNE embedding vectors?
• RQ5 Can the spectrally-reduced graphs achieve both high speedup and high-quality solutions in practical graph and data analysis workloads?

Key findings

• The coPapersCiteseer graph (0.43M nodes, 16M edges) was reduced to 13K nodes and 17K edges in approximately 16 seconds, achieving a 32X reduction in nodes and 950X reduction in edges.
• Spectral graph partitioning using the reduced graph achieved up to 1100X speedup compared to the original graph, with consistently higher partitioning quality than METIS for larger partition counts.
• t-SNE visualization using spectrally-reduced graphs achieved up to 60X speedup while preserving clear cluster boundaries and high-quality embeddings.
• Strong correlations (up to 0.95) were observed between t-SNE embedding vectors and the first few Laplacian eigenvectors, especially when including the 10th eigenvector, which aligns with the ground-truth number of clusters (10) in USPS and MNIST data sets.
• The spectrally-reduced graphs maintained high solution quality across all experiments, with minimal loss in accuracy despite massive size reduction.
• The proposed constrained SGD optimization and iterative scaling framework enabled robust spectral preservation and scalability, achieving nearly-linear time complexity.