[Paper Review] Local Augmentation for Graph Neural Networks

Songtao Liu, Ying, Rex+7 more|arXiv (Cornell University)|Sep 8, 2021

Advanced Graph Neural NetworksComputer Science62 references19 citations

TL;DR

This paper proposes Local Augmentation for Graph Neural Networks (LAGNN), a plug-and-play data augmentation framework that generates synthetic neighborhood features conditioned on a central node’s representation via a learned generative model. By injecting these generated features during training, LAGNN enhances GNN performance, achieving state-of-the-art results with average improvements of 3.4% and 1.6% in test accuracy on Cora, Citeseer, and PubMed over GCN and GAT respectively.

ABSTRACT

Graph Neural Networks (GNNs) have achieved remarkable performance on graph-based tasks. The key idea for GNNs is to obtain informative representation through aggregating information from local neighborhoods. However, it remains an open question whether the neighborhood information is adequately aggregated for learning representations of nodes with few neighbors. To address this, we propose a simple and efficient data augmentation strategy, local augmentation, to learn the distribution of the node features of the neighbors conditioned on the central node's feature and enhance GNN's expressive power with generated features. Local augmentation is a general framework that can be applied to any GNN model in a plug-and-play manner. It samples feature vectors associated with each node from the learned conditional distribution as additional input for the backbone model at each training iteration. Extensive experiments and analyses show that local augmentation consistently yields performance improvement when applied to various GNN architectures across a diverse set of benchmarks. For example, experiments show that plugging in local augmentation to GCN and GAT improves by an average of 3.4\% and 1.6\% in terms of test accuracy on Cora, Citeseer, and Pubmed. Besides, our experimental results on large graphs (OGB) show that our model consistently improves performance over backbones. Code is available at https://github.com/SongtaoLiu0823/LAGNN.

Motivation & Objective

• To address the limited expressive power of GNNs on low-degree nodes with sparse neighborhood connectivity.
• To improve node representation learning by enriching local neighborhood information beyond the actual neighbors.
• To develop a general, plug-and-play data augmentation framework compatible with any GNN architecture.
• To overcome the limitations of global-level data augmentation by focusing on local, node-conditioned feature generation.
• To alleviate over-smoothing by preserving locality through enhanced neighborhood feature diversity.

Proposed method

• A conditional generative model is pre-trained to learn the distribution of neighbor node features given the central node’s feature.
• At each training iteration, the model samples synthetic feature vectors from the learned conditional distribution for the central node’s neighborhood.
• The generated features are concatenated with the original features and fed into the backbone GNN model.
• The generative model is decoupled from the GNN training, enabling plug-and-play integration with any GNN architecture.
• The method leverages a variational autoencoder-like framework to model the conditional distribution of neighbor features.
• The approach is applied end-to-end in a self-supervised manner without requiring additional labels.

Experimental results

Research questions

• RQ1 Can local data augmentation that generates synthetic neighbor features improve GNN performance on low-degree nodes?
• RQ2 Does conditioning the generation of neighbor features on the central node’s representation lead to better node representations than global augmentation?
• RQ3 Can a plug-and-play augmentation framework consistently improve diverse GNN architectures across multiple benchmarks?
• RQ4 How does local augmentation affect over-smoothing in deep GNNs?
• RQ5 Does the method generalize to large-scale graphs, such as those in the Open Graph Benchmark (OGB)?

Key findings

• LAGNN improves test accuracy by an average of 3.4% over GCN and 1.6% over GAT on Cora, Citeseer, and PubMed.
• On Pubmed, LAGNN achieves a 1.7% improvement in test accuracy for nodes with degrees in [2,5], and 0.2% for nodes in [6,20].
• The method outperforms existing topology-level and feature-level augmentation baselines such as DropEdge, G-GNN, and G aug on semi-supervised node classification tasks.
• The MADgap metric shows that LAGCN maintains or improves representation diversity across layers, indicating reduced over-smoothing.
• The framework is effective across multiple GNN architectures, including GCN, GAT, and others, demonstrating broad compatibility and generalization.
• The ablation study confirms that the conditional generation of neighbor features is essential, and random or unconditioned augmentation yields inferior results.