[Paper Review] GPS: Graph Contrastive Learning via Multi-scale Augmented Views from Adversarial Pooling

Ju Wei, Yiyang Gu+6 more|arXiv (Cornell University)|Jan 29, 2024

Data-Driven Disease SurveillanceMedicine6 citations

TL;DR

GPS leverages learnable graph pooling to automatically generate multi-scale positive views for graph contrastive learning, using adversarial and consistency-based objectives to improve unsupervised graph representations.

ABSTRACT

Self-supervised graph representation learning has recently shown considerable promise in a range of fields, including bioinformatics and social networks. A large number of graph contrastive learning approaches have shown promising performance for representation learning on graphs, which train models by maximizing agreement between original graphs and their augmented views (i.e., positive views). Unfortunately, these methods usually involve pre-defined augmentation strategies based on the knowledge of human experts. Moreover, these strategies may fail to generate challenging positive views to provide sufficient supervision signals. In this paper, we present a novel approach named Graph Pooling ContraSt (GPS) to address these issues. Motivated by the fact that graph pooling can adaptively coarsen the graph with the removal of redundancy, we rethink graph pooling and leverage it to automatically generate multi-scale positive views with varying emphasis on providing challenging positives and preserving semantics, i.e., strongly-augmented view and weakly-augmented view. Then, we incorporate both views into a joint contrastive learning framework with similarity learning and consistency learning, where our pooling module is adversarially trained with respect to the encoder for adversarial robustness. Experiments on twelve datasets on both graph classification and transfer learning tasks verify the superiority of the proposed method over its counterparts.

Motivation & Objective

• Motivate better positive-view generation for graph contrastive learning without manual augmentations.
• Propose a dual-view (strongly-augmented and weakly-augmented) framework using learnable graph pooling.
• Incorporate adversarial training to improve robustness of augmentations.
• Combine similarity learning on weak views with soft consistency learning on strong views.
• Empirically validate GPS on graph classification and transfer learning across diverse datasets.

Proposed method

• Use a 2-layer GIN as the graph encoder with a readout to obtain graph-level representations.
• Introduce two learnable graph pooling modules (for strong and weak augmentations) to create multi-scale views: TopK-based pooling and cluster-based pooling.
• Generate two augmented graphs with pooling ratios rho1 (strong) and rho2 (weak) to provide complementary views.
• Apply adversarial learning where the weak-augmentation pooler is trained against the encoder via a minimax objective.
• Implement similarity learning on weak views via a predictor and a target encoder (BYOL-like setup).
• Implement consistency learning on strong views by aligning distributions of similarities across the batch using KL-divergence (soft constraints).
• Update pooling modules adversarially (gradient ascent) with respect to the encoder, while updating the encoder to minimize the losses.

Experimental results

Research questions

• RQ1 Can learnable multi-scale graph pooling provide effective, dataset-adaptive positive views for graph contrastive learning?
• RQ2 Do strongly-augmented and weakly-augmented views capture complementary semantic information that improves representations when combined?
• RQ3 Does adversarial training of pooling modules improve robustness and performance of graph representations in unsupervised settings?
• RQ4 How does GPS perform on graph classification and transfer learning tasks compared with state-of-the-art unsupervised baselines?

Key findings

• GPS achieves superior performance across twelve datasets for graph classification and transfer learning compared with unsupervised baselines.
• GPS-TopK and GPS-Cluster variants reach top-tier or competitive results across datasets, validating learnable pooling for augmented views.
• Ablation studies show both weak and strong augmentations and adversarial training contribute to performance gains.
• Soft consistency learning (KL-based) improves use of strongly-augmented views by preserving semantic structure.
• Graph pooling ratios are generally robust, with performance not highly sensitive to rho choices.
• Clustering-based evaluation shows GPS achieves strong graph-level discrimination on multiple benchmarks.