[Paper Review] Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Xin Liu, Mingyu Yan+6 more|arXiv (Cornell University)|Feb 10, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This survey presents a comprehensive, algorithmic perspective on graph neural network (GNN) acceleration, introducing a novel five-category taxonomy that classifies methods by optimized factors and core mechanisms. It systematically reviews graph-level and model-level optimizations—such as sampling, sparsification, partitioning, simplification, and compression—highlighting their efficiency, trade-offs, and synergies, and identifies future research directions in dynamic graph support, hardware-friendly algorithms, and algorithm-hardware co-design.

ABSTRACT

Graph neural networks (GNNs) have been a hot spot of recent research and are widely utilized in diverse applications. However, with the use of huger data and deeper models, an urgent demand is unsurprisingly made to accelerate GNNs for more efficient execution. In this paper, we provide a comprehensive survey on acceleration methods for GNNs from an algorithmic perspective. We first present a new taxonomy to classify existing acceleration methods into five categories. Based on the classification, we systematically discuss these methods and highlight their correlations. Next, we provide comparisons from aspects of the efficiency and characteristics of these methods. Finally, we suggest some promising prospects for future research.

Motivation & Objective

• Address the urgent need for efficient GNN execution due to increasing graph scale and model depth.
• Systematically classify existing algorithmic acceleration methods for GNNs into a novel five-category taxonomy.
• Compare the efficiency, characteristics, and interrelations of acceleration techniques across training and inference.
• Identify key challenges and future research directions in GNN acceleration, especially for dynamic graphs and hardware co-design.
• Bridge the gap between algorithmic optimization and hardware-aware design to enhance GNN framework performance.

Proposed method

• Propose a double-level taxonomy that categorizes GNN acceleration methods based on optimized factors (e.g., computation, memory, communication) and core mechanisms (e.g., sampling, sparsification, partitioning).
• Review and analyze five categories of algorithmic acceleration: sampling, sparsification, partitioning, simplification, and compression, with emphasis on their underlying principles and design goals.
• Evaluate methods based on key metrics: computation cost (Compt.), memory access (Mem.), and communication cost (Com.), particularly in distributed or edge settings.
• Highlight correlations among methods, such as how sparsification reduces memory access and sampling reduces computation, and how they can be combined.
• Introduce hardware-aware optimization strategies, such as locality-aware sampling, to align algorithmic choices with underlying hardware features.
• Advocate for algorithm-hardware co-design, using insights from graph processing systems like GraphDynS as a blueprint for co-optimizing GNN execution semantics and hardware datapaths.

Experimental results

Research questions

• RQ1 How can existing algorithmic acceleration methods for GNNs be systematically classified based on their optimization targets and mechanisms?
• RQ2 What are the key trade-offs and performance characteristics of different GNN acceleration techniques in terms of computation, memory, and communication costs?
• RQ3 How do sampling, sparsification, partitioning, simplification, and compression methods relate to one another in terms of design principles and performance outcomes?
• RQ4 What are the major challenges in accelerating GNNs on dynamic graphs, and how can current methods be extended to support dynamic topologies and features?
• RQ5 How can algorithmic design be aligned with hardware features to achieve greater efficiency, and what role does co-design between algorithms and hardware play in future GNN acceleration?

Key findings

• The proposed five-category taxonomy—sampling, sparsification, partitioning, simplification, and compression—provides a unified framework for classifying and comparing GNN acceleration methods.
• Sampling methods such as AS-GCN and GraphSAINT reduce computation cost by selecting representative neighbors, achieving significant speedups with controlled variance.
• Sparsification techniques like NeuralSparse remove task-irrelevant edges, reducing memory access and improving generalization through implicit denoising.
• Partitioning methods improve communication efficiency in distributed settings by minimizing cross-partition edges and enabling load balancing across devices.
• Compression methods such as KD-GCN extend knowledge distillation to dynamic graphs, enabling efficient model distillation with virtual edge injection.
• Future research should prioritize hardware-friendly algorithm design and algorithm-hardware co-design, as current methods often overlook low-level hardware characteristics, especially in dynamic graph scenarios.