[论文解读] FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

Jie Chen, Tengfei Ma+1 more|arXiv (Cornell University)|Jan 30, 2018

Advanced Graph Neural Networks被引用 639

一句话总结

FastGCN 将图卷积重新表述为积分变换，使蒙特卡洛采样结合重要性重加权来进行自归纳学习，并使每批次成本远低于标准 GCNs 和 GraphSAGE，同时保持可比的准确性。

ABSTRACT

The graph convolutional networks (GCN) recently proposed by Kipf and Welling are an effective graph model for semi-supervised learning. This model, however, was originally designed to be learned with the presence of both training and test data. Moreover, the recursive neighborhood expansion across layers poses time and memory challenges for training with large, dense graphs. To relax the requirement of simultaneous availability of test data, we interpret graph convolutions as integral transforms of embedding functions under probability measures. Such an interpretation allows for the use of Monte Carlo approaches to consistently estimate the integrals, which in turn leads to a batched training scheme as we propose in this work---FastGCN. Enhanced with importance sampling, FastGCN not only is efficient for training but also generalizes well for inference. We show a comprehensive set of experiments to demonstrate its effectiveness compared with GCN and related models. In particular, training is orders of magnitude more efficient while predictions remain comparably accurate.

研究动机与目标

• Introduce an inductive learning framework for graph convolutions that removes dependence on test data.
• Reduce training time and memory load for GCNs on large, dense graphs via sampling.
• Provide a principled Monte Carlo estimator for layer-wise convolutions and loss.
• Develop variance-reduction techniques to improve sampling efficiency while preserving accuracy.

提出的方法

• Interpret graph convolutions as integral transforms of vertex embedding functions under a probability measure.
• Use Monte Carlo sampling to estimate layer transforms and the loss, enabling batched training.
• Apply importance sampling to reduce estimator variance and derive an optimal sampling distribution.
• Provide a batched training algorithm (FastGCN) with sampling per layer and backpropagation of batch gradients.
• Propose two variants: uniform sampling and importance sampling; compute precomputation of AH(0) to speed training.
• Show how the method reduces memory footprint by sampling vertices rather than entire neighborhoods.

实验结果

研究问题

• RQ1 Can graph convolutions be reformulated as integral transforms to enable inductive learning without requiring test data?
• RQ2 Does vertex-wise sampling (as opposed to neighborhood sampling) yield substantial computational savings for GCNs on large graphs?
• RQ3 Can importance sampling improve the variance of Monte Carlo estimates in FastGCN, compared to uniform sampling?
• RQ4 How does FastGCN perform in terms of training time and predictive accuracy compared with GCN and GraphSAGE on standard benchmarks?
• RQ5 What are the practical implications for inference when using an inductive FastGCN framework?

主要发现

• FastGCN achieves order-of-magnitude speedups over GCN and GraphSAGE in per-batch training time.
• Importance sampling consistently yields higher accuracy than uniform sampling in experiments.
• Prediction accuracies with FastGCN are highly comparable to those of GCN and GraphSAGE across datasets.
• Precomputing AH(0) for the input layer can substantially reduce training time with comparable accuracy.
• FastGCN enables inductive learning by separating training/test data and avoiding full neighborhood expansion.