[Paper Review] InfoGCL: Information-Aware Graph Contrastive Learning

Dongkuan Xu, Wei Cheng+3 more|arXiv (Cornell University)|Oct 28, 2021

Advanced Graph Neural NetworksComputer Science44 references32 citations

TL;DR

InfoGCL provides an information-theoretic framework for graph contrastive learning that decouples view augmentation, view encoding, and representation contrasting, unifying prior methods and achieving strong graph and node classification performance.

ABSTRACT

Various graph contrastive learning models have been proposed to improve the performance of learning tasks on graph datasets in recent years. While effective and prevalent, these models are usually carefully customized. In particular, although all recent researches create two contrastive views, they differ greatly in view augmentations, architectures, and objectives. It remains an open question how to build your graph contrastive learning model from scratch for particular graph learning tasks and datasets. In this work, we aim to fill this gap by studying how graph information is transformed and transferred during the contrastive learning process and proposing an information-aware graph contrastive learning framework called InfoGCL. The key point of this framework is to follow the Information Bottleneck principle to reduce the mutual information between contrastive parts while keeping task-relevant information intact at both the levels of the individual module and the entire framework so that the information loss during graph representation learning can be minimized. We show for the first time that all recent graph contrastive learning methods can be unified by our framework. We empirically validate our theoretical analysis on both node and graph classification benchmark datasets, and demonstrate that our algorithm significantly outperforms the state-of-the-arts.

Motivation & Objective

• Motivate how graph information is transformed and transferred during contrastive learning on graphs.
• Propose an information-theoretic framework (InfoGCL) to minimize information loss while preserving task-relevant information at module and framework levels.
• Unify existing graph contrastive learning methods under a common principle and provide practical guidelines for module design.
• Analyze the role of negative samples in graph contrastive learning and assess when negatives are beneficial.

Proposed method

• Decompose graph contrastive learning into three stages: view augmentation, view encoding, and representation contrasting.
• Ground each stage in Information Bottleneck principles to derive optimality corollaries for views, encoders, and contrastive modes.
• Define optimal augmented views (minimize I(v_i; v_j) while keeping I(v_i; y) = I(v_j; y) = I(G; y)).
• Define optimal view encoders (minimize I(f_i(v_i); v_i) subject to I(f_i(v_i); v_j) = I(v_i; v_j)).
• Define optimal contrastive modes (select c_i, c_j to maximize retained task-relevant information, via a mutual-information-based criterion).
• Evaluate a range of graph view augmentations (node dropping, edge perturbation, attribute masking, subgraph sampling) and contrastive modes (global-global, local-global, local-local, multi-scale, hybrid).
• Investigate whether negative samples are necessary by adopting a SimSiam-style loss without negatives and compare with negative-inclusive variants.

Experimental results

Research questions

• RQ1 How should augmented views be selected to maximize task-relevant information while minimizing shared non-task information?
• RQ2 What encoders most effectively preserve shared, task-relevant information after encoding?
• RQ3 Which contrastive mode best preserves downstream task information given optimal views and encoders?
• RQ4 Do negative samples meaningfully affect performance in graph contrastive learning across tasks and datasets?

Key findings

• InfoGCL achieves competitive to state-of-the-art results on node and graph classification benchmarks.
• On graph classification, InfoGCL matches or exceeds leading unsupervised and competitive supervised methods, e.g., achieving 91.2% on MUTAG, 63.5% on Citeseer, 75.1% on Pubmed (examples from Table 2).
• InfoGCL outperforms many baselines across multiple datasets, with notable gains on graph classification tasks (e.g., up to ≈5.2% relative improvement in some settings).
• Negative samples are not universally necessary; removing negatives preserves performance on several graph datasets but may slightly hurt node-level tasks, particularly on sparser graphs like Cora/Citeseer/Pubmed (Table 4).
• A unified view is possible: most recent graph contrastive methods can be interpreted as instances of the InfoGCL three-stage framework guided by the Information Bottleneck.