[Paper Review] GraphZoom: A multi-level spectral approach for accurate and scalable graph embedding

Chenhui Deng, Zhiqiang Zhao+3 more|arXiv (Cornell University)|Oct 6, 2019

Advanced Graph Neural Networks29 references55 citations

TL;DR

GraphZoom introduces a four-phase, multi-level framework that fuses node attributes with topology, spectrally coarsens graphs, applies standard unsupervised embeddings on the coarsest level, and refines embeddings via graph filtering to improve accuracy and scalability.

ABSTRACT

Graph embedding techniques have been increasingly deployed in a multitude of different applications that involve learning on non-Euclidean data. However, existing graph embedding models either fail to incorporate node attribute information during training or suffer from node attribute noise, which compromises the accuracy. Moreover, very few of them scale to large graphs due to their high computational complexity and memory usage. In this paper we propose GraphZoom, a multi-level framework for improving both accuracy and scalability of unsupervised graph embedding algorithms. GraphZoom first performs graph fusion to generate a new graph that effectively encodes the topology of the original graph and the node attribute information. This fused graph is then repeatedly coarsened into much smaller graphs by merging nodes with high spectral similarities. GraphZoom allows any existing embedding methods to be applied to the coarsened graph, before it progressively refine the embeddings obtained at the coarsest level to increasingly finer graphs. We have evaluated our approach on a number of popular graph datasets for both transductive and inductive tasks. Our experiments show that GraphZoom can substantially increase the classification accuracy and significantly accelerate the entire graph embedding process by up to 40.8x, when compared to the state-of-the-art unsupervised embedding methods.

Motivation & Objective

• Motivate and address the trade-off between embedding accuracy and scalability in graph embedding.
• Propose a unified multi-level framework that preserves spectral properties during coarsening.
• Enable plug-and-play use of any unsupervised embedding method within the coarsening framework.
• Improve embedding quality through a graph-filter-based refinement stage.
• Demonstrate scalability to large graphs while maintaining or boosting accuracy.

Proposed method

• Phase 1 graph fusion creates a fused adjacency by combining topology and a kNN attribute graph built from node features with cosine similarity, controlled by a fusion weight beta.
• Phase 2 spectral coarsening builds a hierarchy of coarsened graphs using local spectral embedding and a spectral node affinity measure to merge spectrally similar nodes, preserving first few Laplacian eigenvectors; coarsening uses a linear-time aggregation scheme and optional GRASS spectral sparsification for density control.
• Phase 3 graph embedding applies any existing unsupervised embedding method to the coarsest graph to obtain initial embeddings.
• Phase 4 embedding refinement maps coarse embeddings to finer graphs via projection operators and applies a low-pass graph filter (approximated with (I-L)^k) to smooth embeddings, iterating to recover embeddings for the original graph.

Experimental results

Research questions

• RQ1 Can a multi-level spectral approach jointly improve embedding accuracy and scalability for unsupervised graph embedding?
• RQ2 Does fusing node attributes with topology in a fused graph enhance embedding quality over topology-only methods?
• RQ3 How effective is spectral coarsening at preserving essential graph structure for subsequent embeddings?
• RQ4 Can a non-training, graph-filter-based refinement stage meaningfully improve final embeddings without additional model training?
• RQ5 Is GraphZoom agnostic to the underlying embedding kernel and scalable to very large graphs?

Key findings

• GraphZoom achieves higher embedding accuracy than baseline unsupervised methods across transductive and inductive tasks, with relative gains up to 19.4%.
• GraphZoom can accelerate the embedding process by up to 40.8x on standard benchmarks compared to state-of-the-art methods.
• On large-scale Friendster data, GraphZoom yields up to 49.9% higher Micro-F1 and up to 119.8x speedup over a baseline.
• The refinement stage operates via efficient sparse matrix multiplications and graph filtering, avoiding expensive inversions while reducing high-frequency noise.
• GraphZoom is highly composable and agnostic to the underlying embedding kernel, improving both accuracy and speed when paired with DeepWalk, node2vec, DGI, or GraphSAGE.
• Ablation analysis indicates the refinement kernel (graph filtering) substantially improves results, especially with larger coarsening.