[Paper Review] Adversarial Graph Augmentation to Improve Graph Contrastive Learning

Susheel Suresh, Li Pan+2 more|arXiv (Cornell University)|Jun 10, 2021

Advanced Graph Neural Networks80 references141 citations

TL;DR

AD-GCL introduces a learnable, adversarial graph data augmentation strategy for graph contrastive learning, optimizing augmentations to remove redundant information and improve downstream performance across unsupervised, transfer, and semi-supervised tasks.

ABSTRACT

Self-supervised learning of graph neural networks (GNN) is in great need because of the widespread label scarcity issue in real-world graph/network data. Graph contrastive learning (GCL), by training GNNs to maximize the correspondence between the representations of the same graph in its different augmented forms, may yield robust and transferable GNNs even without using labels. However, GNNs trained by traditional GCL often risk capturing redundant graph features and thus may be brittle and provide sub-par performance in downstream tasks. Here, we propose a novel principle, termed adversarial-GCL (AD-GCL), which enables GNNs to avoid capturing redundant information during the training by optimizing adversarial graph augmentation strategies used in GCL. We pair AD-GCL with theoretical explanations and design a practical instantiation based on trainable edge-dropping graph augmentation. We experimentally validate AD-GCL by comparing with the state-of-the-art GCL methods and achieve performance gains of up-to $14\%$ in unsupervised, $6\%$ in transfer, and $3\%$ in semi-supervised learning settings overall with 18 different benchmark datasets for the tasks of molecule property regression and classification, and social network classification.

Motivation & Objective

• Motivate self-supervised graph representation learning to address label scarcity.
• Develop a principled AD-GCL framework that pairs a GNN encoder with a learnable augmentation strategy.
• Theoretically justify how adversarial augmentation can bound task-relevant information while reducing redundancy.
• Empirically validate AD-GCL across 18 datasets for molecule and social-network tasks in unsupervised, transfer, and semi-supervised settings.

Proposed method

• Formulate AD-GCL as a min-max game: min over augmentations T in a family T and max over encoder f of I(f(G);f(t(G))).
• Instantiate T via learnable edge dropping where each edge e has drop probability ω_e determined by a GNN augmenter.
• Parameterize ω_e with a GNN, using a Gumbel-softmax relaxation to sample edge drops during training.
• Regularize the augmentation to avoid overly aggressive perturbations by penalizing the average drop probability per graph.
• Estimate the mutual information term with InfoNCE in mini-batches to optimize the objective.
• Provide theoretical insights tying AD-GCL to information bottleneck principles and 1-WL expressiveness.

Experimental results

Research questions

• RQ1 Can an adversarially learned augmentation outperform fixed or manually designed augmentations in graph contrastive learning?
• RQ2 How does optimizing the augmentation strategy influence the amount of task-relevant versus redundant information captured by GNN encoders?
• RQ3 Does AD-GCL improve robustness and transferability of graph representations without access to downstream labels?
• RQ4 What is the impact of regularization on the augmentation search space on downstream performance?

Key findings

• AD-GCL achieves notable gains over state-of-the-art baselines in unsupervised, transfer, and semi-supervised settings across 18 datasets.
• In unsupervised tasks, AD-GCL improves performance up to 14% relative to baselines.
• In transfer and semi-supervised tasks, AD-GCL shows consistent improvements up to about 6% and 3%, respectively.
• Edge-dropping augmentation learned by AD-GCL outperforms non-adversarial (random) edge dropping and predefined augmentation strategies.
• Regularization of the augmentation search space (λ_reg) is beneficial, and the OPT variant can offer robustness across datasets.
• The learned input-graph-dependent augmentation maintains information relevant to downstream tasks while reducing redundant information, aligning with information bottleneck intuition.