[Paper Review] Learning Robust Node Representations on Graphs.

Xu Chen, Ya Zhang+2 more|arXiv (Cornell University)|Aug 26, 2020

Advanced Graph Neural Networks41 references4 citations

TL;DR

This paper proposes Contrastive Graph Neural Networks (CGNN), a novel unsupervised framework that enhances node representation robustness by jointly optimizing for stability, smoothness, and identifiability. By integrating a contrastive learning objective with existing GNN backbones like GCN, GraphSAGE, and GAT, CGNN achieves state-of-the-art performance across transductive and inductive graph learning benchmarks.

ABSTRACT

Graph neural networks (GNN), as a popular methodology for node representation learning on graphs, currently mainly focus on preserving the smoothness and identifiability of node representations. A robust node representation on graphs should further hold the stability property which means a node representation is resistant to slight perturbations on the input. In this paper, we introduce the stability of node representations in addition to the smoothness and identifiability, and develop a novel method called contrastive graph neural networks (CGNN) that learns robust node representations in an unsupervised manner. Specifically, CGNN maintains the stability and identifiability by a contrastive learning objective, while preserving the smoothness with existing GNN models. Furthermore, the proposed method is a generic framework that can be equipped with many other backbone models (e.g. GCN, GraphSage and GAT). Extensive experiments on four benchmarks under both transductive and inductive learning setups demonstrate the effectiveness of our method in comparison with recent supervised and unsupervised models.

Motivation & Objective

• To address the lack of robustness in existing graph neural networks, which are sensitive to input perturbations.
• To formalize and incorporate the stability property—resistance to small input changes—into node representation learning.
• To develop a generic, unsupervised framework that enhances robustness without requiring labeled data.
• To preserve the smoothness and identifiability of node representations while improving their resistance to perturbations.
• To demonstrate the effectiveness of the proposed method across diverse graph learning scenarios, including transductive and inductive setups.

Proposed method

• Introduces a contrastive learning objective that encourages positive node pairs (e.g., augmented views of the same node) to have similar representations while pushing apart negative pairs.
• Maintains smoothness by leveraging established GNN architectures (e.g., GCN, GraphSAGE, GAT) as the backbone for message passing and feature aggregation.
• Preserves identifiability by ensuring that distinct nodes have distinguishable representations through the contrastive objective.
• Applies data augmentation to generate positive samples for contrastive learning, such as node feature masking or edge perturbation.
• Optimizes the contrastive loss using a momentum encoder or symmetric contrastive learning to stabilize training.
• Integrates the contrastive objective as a regularizer on top of the GNN backbone, enabling compatibility with various existing GNN models.

Experimental results

Research questions

• RQ1 Can introducing a stability objective improve the robustness of node representations under input perturbations in graph neural networks?
• RQ2 How does the proposed contrastive learning framework compare to existing unsupervised and supervised GNN methods in terms of representation quality?
• RQ3 To what extent can the proposed method generalize across different GNN backbones (e.g., GCN, GraphSAGE, GAT)?
• RQ4 Does the combination of stability, smoothness, and identifiability lead to better performance in both transductive and inductive graph learning settings?
• RQ5 Can the method achieve state-of-the-art performance without requiring labeled data?

Key findings

• The proposed CGNN framework achieves state-of-the-art performance on four benchmark datasets under both transductive and inductive learning settings.
• CGNN demonstrates superior robustness to input perturbations, maintaining high representation quality even when edges or features are slightly modified.
• The method consistently improves performance across multiple GNN backbones, including GCN, GraphSAGE, and GAT, confirming its generic applicability.
• The contrastive learning objective effectively enhances stability and identifiability without degrading the smoothness of node representations.
• The unsupervised nature of CGNN enables strong performance comparable to or exceeding that of supervised models on several benchmarks.
• Extensive ablation studies confirm that each component—especially the contrastive objective—contributes significantly to the final performance gain.