[Paper Review] Deep Graph Contrastive Representation Learning

Yanqiao Zhu, Yichen Xu+4 more|arXiv (Cornell University)|Jun 7, 2020

Advanced Graph Neural NetworksComputer Science44 references412 citations

TL;DR

GRACE learns unsupervised node embeddings by contrasting node representations across two corrupted graph views, improving over prior methods and even rivaling supervised approaches on transductive tasks.

ABSTRACT

Graph representation learning nowadays becomes fundamental in analyzing graph-structured data. Inspired by recent success of contrastive methods, in this paper, we propose a novel framework for unsupervised graph representation learning by leveraging a contrastive objective at the node level. Specifically, we generate two graph views by corruption and learn node representations by maximizing the agreement of node representations in these two views. To provide diverse node contexts for the contrastive objective, we propose a hybrid scheme for generating graph views on both structure and attribute levels. Besides, we provide theoretical justification behind our motivation from two perspectives, mutual information and the classical triplet loss. We perform empirical experiments on both transductive and inductive learning tasks using a variety of real-world datasets. Experimental experiments demonstrate that despite its simplicity, our proposed method consistently outperforms existing state-of-the-art methods by large margins. Moreover, our unsupervised method even surpasses its supervised counterparts on transductive tasks, demonstrating its great potential in real-world applications.

Motivation & Objective

• Motivate unsupervised graph representation learning without relying on graph proximity reconstructions or injective readout constraints.
• Propose a node-level contrastive framework that maximizes agreement across two corrupted graph views.
• Develop dual graph view generation via topology and attribute corruption (edge removal and feature masking).
• Provide theoretical connections to mutual information (InfoMax) and triplet loss.
• Empirically validate on transductive and inductive node classification across multiple datasets.

Proposed method

• Use a GNN encoder to produce node embeddings from two corrupted graph views.
• Generate two views by removing edges (structure corruption) and masking features (attribute corruption).
• Apply a contrastive loss that pulls corresponding node embeddings across views together and pushes apart all other node embeddings.
• Use a two-layer MLP projection g to obtain a discriminative score between node pairs.
• Optimize the average contrastive objective across all nodes, without relying on an explicit graph-level readout.

Experimental results

Research questions

• RQ1 Can node-level contrastive learning on two graph views yield strong unsupervised node representations?
• RQ2 Do dual corruption strategies (structure and feature masking) provide diverse contexts to improve learning?
• RQ3 How does GRACE relate to mutual information maximization and the triplet loss framework?
• RQ4 How does GRACE perform on standard transductive and inductive graph datasets compared to existing unsupervised and supervised methods?

Key findings

• GRACE achieves state-of-the-art or competitive results among unsupervised methods across six datasets.
• On transductive tasks, GRACE surpasses DGI and other baselines and can rival supervised models (e.g., GCN, SGC) on several datasets.
• On inductive tasks, GRACE outperforms most baselines and approaches or matches supervised performance on Reddit and PPI datasets.
• Theoretical analysis shows GRACE maximizes a lower bound on mutual information between input features and node embeddings in two views and relates to the triplet loss.
• GRACE remains robust to sparse features and benefits from corruption at both topology and feature levels.