[Paper Review] Label Contrastive Coding based Graph Neural Network for Graph Classification

Yuxiang Ren, Jiyang Bai+1 more|arXiv (Cornell University)|Jan 14, 2021

Advanced Graph Neural Networks36 references4 citations

TL;DR

This paper proposes LCGNN, a novel graph neural network that enhances graph classification by integrating label contrastive coding with supervised learning. By leveraging a dynamic label memory bank and momentum-updated encoder, LCGNN improves instance-level intra-class compactness and inter-class separability, achieving state-of-the-art performance on seven of eight benchmark datasets and outperforming baselines with less training data.

ABSTRACT

Graph classification is a critical research problem in many applications from different domains. In order to learn a graph classification model, the most widely used supervision component is an output layer together with classification loss (e.g.,cross-entropy loss together with softmax or margin loss). In fact, the discriminative information among instances are more fine-grained, which can benefit graph classification tasks. In this paper, we propose the novel Label Contrastive Coding based Graph Neural Network (LCGNN) to utilize label information more effectively and comprehensively. LCGNN still uses the classification loss to ensure the discriminability of classes. Meanwhile, LCGNN leverages the proposed Label Contrastive Loss derived from self-supervised learning to encourage instance-level intra-class compactness and inter-class separability. To power the contrastive learning, LCGNN introduces a dynamic label memory bank and a momentum updated encoder. Our extensive evaluations with eight benchmark graph datasets demonstrate that LCGNN can outperform state-of-the-art graph classification models. Experimental results also verify that LCGNN can achieve competitive performance with less training data because LCGNN exploits label information comprehensively.

Motivation & Objective

• To address the limitation of standard classification loss in capturing fine-grained instance-level discriminative features in graph classification.
• To extend contrastive learning from unsupervised to supervised settings by incorporating label information into positive pair selection.
• To improve model generalization and data efficiency by jointly optimizing classification loss and label contrastive loss.
• To develop a dynamic label memory bank and momentum-updated encoder to stabilize contrastive learning in graph representation learning.
• To enable end-to-end training that effectively utilizes label information at both instance and class levels.

Proposed method

• LCGNN introduces a Label Contrastive Loss that treats all instances of the same label as multiple positive pairs, promoting intra-class compactness and inter-class separability.
• A dynamic label memory bank stores encoded representations of labeled graphs, enabling consistent and scalable contrastive learning across training steps.
• A momentum-updated encoder generates target embeddings for contrastive learning, stabilizing the training process and improving representation consistency.
• The model jointly optimizes a standard cross-entropy classification loss and the proposed label contrastive loss to balance class discriminability and instance-level feature learning.
• The framework supports end-to-end training, allowing the contrastive component to be directly aligned with the downstream classification objective.
• The method is compatible with any GNN backbone, such as GIN, and is evaluated using standard graph classification benchmarks.

Experimental results

Research questions

• RQ1 Can label information be effectively leveraged in contrastive learning to improve graph representation quality beyond standard classification loss?
• RQ2 How does label contrastive coding compare to standard contrastive learning that treats each instance as a unique class?
• RQ3 Can the proposed method achieve better performance with less labeled data by exploiting label information more comprehensively?
• RQ4 What is the impact of hyperparameters such as contrastive loss weight and momentum on model performance?
• RQ5 Does the dynamic memory bank and momentum encoder mechanism improve training stability and representation consistency in graph contrastive learning?

Key findings

• LCGNN achieves state-of-the-art performance on seven out of eight benchmark graph classification datasets, surpassing existing GNNs.
• On the IMDB-BINARY dataset, LCGNN with GIN backbone achieves 76.1% accuracy, outperforming the baseline GIN model.
• LCGNN maintains superior performance even when training data is reduced to 60%, while the baseline GIN requires 70% to match its performance.
• The ablation study confirms that label contrastive coding significantly improves performance over InfoNCE loss, which underperforms even the baseline GIN model.
• The optimal contrastive loss weight β is found in the range [0.5, 1.0], with performance peaking around β = 0.6–0.8, indicating a trade-off between instance-level and class-level learning.
• The momentum term has a minimal impact on performance, suggesting the method is stable even without momentum when dataset size is moderate.