[Paper Review] Contrastive Multi-View Representation Learning on Graphs

Kaveh Hassani, Amir Hosein Khasahmadi|arXiv (Cornell University)|Jun 10, 2020

Advanced Graph Neural Networks51 references381 citations

TL;DR

This paper presents a self-supervised method that learns node and graph representations by contrasting two structural views of graphs (adjacency and diffusion) and achieves state-of-the-art results on 8/8 node and graph classification benchmarks under linear evaluation.

ABSTRACT

We introduce a self-supervised approach for learning node and graph level representations by contrasting structural views of graphs. We show that unlike visual representation learning, increasing the number of views to more than two or contrasting multi-scale encodings do not improve performance, and the best performance is achieved by contrasting encodings from first-order neighbors and a graph diffusion. We achieve new state-of-the-art results in self-supervised learning on 8 out of 8 node and graph classification benchmarks under the linear evaluation protocol. For example, on Cora (node) and Reddit-Binary (graph) classification benchmarks, we achieve 86.8% and 84.5% accuracy, which are 5.5% and 2.4% relative improvements over previous state-of-the-art. When compared to supervised baselines, our approach outperforms them in 4 out of 8 benchmarks. Source code is released at: https://github.com/kavehhassani/mvgrl

Motivation & Objective

• Motivate learning graph representations without labels using self-supervised contrastive learning across structural views.
• Propose a two-view graph contrastive framework that leverages adjacency and diffusion views to capture local and global structure.
• Demonstrate that two views outperform multiple views and other encodings, and identify components that maximize performance.
• Evaluate on node and graph classification benchmarks under linear evaluation and clustering protocols.

Proposed method

• Generate two congruent structural views of each graph by transforming the adjacency matrix into a diffusion matrix and sub-sampling nodes.
• Use two dedicated GNN encoders (one per view) followed by a shared projection head to obtain node representations.
• Apply a readout/pooling function to obtain graph representations from node embeddings for each view.
• Train with a discriminator that contrasts node representations from one view with the graph representation from the other view and vice versa, maximizing mutual information between cross-view pairs.
• Maximize MI using a chosen MI estimator (NCE, JSD, NT-Xent, or DV) in a deep InfoMax framework.
• Infer representations by combining node and graph embeddings from both views at inference time.

Experimental results

Research questions

• RQ1 Can self-supervised learning on graphs benefit from contrasting across two structural views (adjacency vs diffusion) rather than multiple views or multi-scale encodings?
• RQ2 Which components (MI estimator, contrastive mode, view choice) most influence node and graph classification performance?
• RQ3 Is a simple readout layer sufficient for graph representations compared to pooling methods like DiffPool?
• RQ4 How do two-view contrastive representations compare to supervised baselines on standard node and graph classification benchmarks?

Key findings

• The method achieves state-of-the-art results on 8/8 node and graph classification benchmarks under linear evaluation.
• On Cora node classification, the approach reaches 86.8% accuracy, a 5.5% relative improvement over previous SOTA.
• On Reddit-Binary graph classification, the approach reaches 84.5% accuracy, a 2.4% relative improvement over previous SOTA.
• Compared to supervised baselines, the approach outperforms on some benchmarks (e.g., on Cora and IMDB-Binary, by about 4.5% and 5.3% relative, respectively).
• The best performance is achieved by contrasting encodings from first-order neighbors and a graph diffusion, with two views being superior to more than two views or other encodings.
• A simple graph readout layer yields better results than hierarchical pooling methods like DiffPool, and additional regularization or normalization can hurt performance.