[Paper Review] Efficient Graph Neural Network Inference at Large Scale

Xinyi Gao, Wentao Zhang+4 more|arXiv (Cornell University)|Nov 1, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes Node-Adaptive Inference (NAI), a plug-and-play framework that accelerates large-scale graph neural network inference by dynamically adapting the propagation order per node based on topological structure, reducing redundant computation. With Inception Distillation for multi-scale feature learning, NAI achieves up to 75× speedup on the ogbn-products dataset while maintaining high accuracy under varying latency constraints.

ABSTRACT

Graph neural networks (GNNs) have demonstrated excellent performance in a wide range of applications. However, the enormous size of large-scale graphs hinders their applications under real-time inference scenarios. Although existing scalable GNNs leverage linear propagation to preprocess the features and accelerate the training and inference procedure, these methods still suffer from scalability issues when making inferences on unseen nodes, as the feature preprocessing requires the graph is known and fixed. To speed up the inference in the inductive setting, we propose a novel adaptive propagation order approach that generates the personalized propagation order for each node based on its topological information. This could successfully avoid the redundant computation of feature propagation. Moreover, the trade-off between accuracy and inference latency can be flexibly controlled by simple hyper-parameters to match different latency constraints of application scenarios. To compensate for the potential inference accuracy loss, we further propose Inception Distillation to exploit the multi scale reception information and improve the inference performance. Extensive experiments are conducted on four public datasets with different scales and characteristics, and the experimental results show that our proposed inference acceleration framework outperforms the SOTA graph inference acceleration baselines in terms of both accuracy and efficiency. In particular, the advantage of our proposed method is more significant on larger-scale datasets, and our framework achieves $75 imes$ inference speedup on the largest Ogbn-products dataset.

Motivation & Objective

• Address the scalability challenge of graph neural network (GNN) inference in real-time, latency-sensitive applications such as recommender systems and fraud detection.
• Overcome the limitation of existing linear propagation GNNs that require a fixed, precomputed graph and cannot efficiently handle unseen nodes during inference.
• Reduce redundant computation in feature propagation by dynamically determining the optimal propagation depth per node based on its topological context.
• Enable flexible trade-offs between inference latency and accuracy through tunable hyper-parameters.
• Compensate for potential accuracy degradation from early termination of propagation using a novel multi-scale knowledge distillation technique.

Proposed method

• Propose Node-Adaptive Inference (NAI), a plug-and-play method that computes a personalized propagation order for each node based on its distance to the stationary state of feature smoothing.
• Use a simple global hyper-parameter to control the trade-off between inference speed and accuracy by determining when to stop feature propagation per node.
• Introduce Inception Distillation, a multi-scale knowledge distillation module that leverages features from multiple propagation depths to enrich the student model’s receptive field.
• Apply online distillation with soft labels from multiple teacher models (e.g., f^(1), f^(2), f^(3)) to improve student performance without retraining the entire model.
• Design a non-parameterized, graph-agnostic feature propagation mechanism that avoids the need for joint training of multiple exits, enhancing training stability.
• Utilize topological information (e.g., node degree, neighborhood structure) to guide early termination, avoiding repeated predictions common in CV/NLP early exiting.

Experimental results

Research questions

• RQ1 Can adaptive, node-specific propagation depth reduce redundant computation in GNN inference without sacrificing model accuracy?
• RQ2 How does dynamic propagation order based on topological structure compare to fixed-order propagation in terms of inference efficiency and accuracy?
• RQ3 To what extent can multi-scale knowledge distillation mitigate performance degradation caused by early termination of feature propagation?
• RQ4 Can the proposed method achieve significant speedup on large-scale, real-world graphs while maintaining high accuracy under varying latency constraints?
• RQ5 How does the framework generalize across diverse graph datasets with different scales and topological characteristics?

Key findings

• NAI achieves up to 75× inference speedup on the ogbn-products dataset, demonstrating its effectiveness on the largest-scale graph evaluated.
• The method maintains high classification accuracy across all datasets, with performance degradation minimized through Inception Distillation, especially when using soft labels and an ensemble of 3 teachers.
• Inception Distillation stabilizes performance across different hyper-parameter settings, with optimal results observed when λ ∈ [0.5, 1] for label softening.
• The framework shows superior performance compared to SOTA baselines in both accuracy and efficiency, with advantages becoming more pronounced on larger graphs.
• The adaptive propagation mechanism reduces redundant computation by avoiding unnecessary propagation steps, particularly benefiting high-degree nodes prone to over-smoothing.
• The method is flexible and deployable in real-world inductive settings, enabling efficient online inference on unseen nodes without requiring a fixed graph structure.