[Paper Review] Community-Invariant Graph Contrastive Learning

Shiyin Tan, Dongyuan Li+3 more|arXiv (Cornell University)|May 2, 2024

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes CI-GCL, a learnable graph contrastive learning framework that maintains community structure during augmentation by maximizing spectral changes in both topology and feature spaces. By unifying topology and feature augmentation under a community-invariant constraint via spectral optimization, CI-GCL improves robustness and achieves state-of-the-art performance on 21 benchmark datasets.

ABSTRACT

Graph augmentation has received great attention in recent years for graph contrastive learning (GCL) to learn well-generalized node/graph representations. However, mainstream GCL methods often favor randomly disrupting graphs for augmentation, which shows limited generalization and inevitably leads to the corruption of high-level graph information, i.e., the graph community. Moreover, current knowledge-based graph augmentation methods can only focus on either topology or node features, causing the model to lack robustness against various types of noise. To address these limitations, this research investigated the role of the graph community in graph augmentation and figured out its crucial advantage for learnable graph augmentation. Based on our observations, we propose a community-invariant GCL framework to maintain graph community structure during learnable graph augmentation. By maximizing the spectral changes, this framework unifies the constraints of both topology and feature augmentation, enhancing the model's robustness. Empirical evidence on 21 benchmark datasets demonstrates the exclusive merits of our framework. Code is released on Github (https://github.com/ShiyinTan/CI-GCL.git).

Motivation & Objective

• To address the limitation of existing graph contrastive learning methods that disrupt community structures during random or knowledge-based augmentation.
• To unify topology and feature augmentation under a single, robust constraint that preserves high-level graph community structure.
• To develop a learnable augmentation framework that is adaptive, differentiable, and efficient for diverse graph datasets.
• To theoretically and empirically validate that maximizing spectral changes preserves community invariance across both topology and feature spaces.

Proposed method

• The framework maximizes spectral changes in the graph Laplacian to enforce community invariance during topology augmentation, leveraging the negative correlation between spectral changes and community disruption.
• For feature augmentation, the method transforms the feature matrix into a symmetric bipartite matrix using co-clustering, enabling spectral-based constraints on feature perturbations.
• A unified spectral change loss is applied to both topology and bipartite feature augmentation, ensuring community structure preservation.
• The method uses the lowest eigenvalues of the normalized Laplacian to constrain community structure, with theoretical proof linking eigenvalues to community partitioning.
• The framework is end-to-end differentiable and supports efficient back-propagation, enabling joint optimization of augmentation and representation learning.
• Community invariance is enforced by maximizing changes in the spectrum, which indirectly preserves the underlying community structure during data augmentation.

Experimental results

Research questions

• RQ1 Can maximizing spectral changes during graph augmentation preserve the underlying community structure in both topology and feature spaces?
• RQ2 How can a unified constraint be designed to simultaneously handle topology and feature augmentation in a learnable graph contrastive learning framework?
• RQ3 Does enforcing community invariance through spectral optimization improve model robustness and generalization across diverse graph datasets?
• RQ4 Can spectral-based constraints be effectively extended from topology to feature augmentation via bipartite co-clustering?

Key findings

• CI-GCL achieves state-of-the-art performance on 21 benchmark graph datasets, outperforming existing methods in node classification and link prediction tasks.
• The framework demonstrates superior robustness against various types of noise due to its unified, community-invariant constraint on both topology and feature augmentation.
• Empirical results show that maximizing spectral changes correlates strongly with preserved community structure, validating the core theoretical assumption.
• The method achieves consistent improvements across diverse datasets, including citation networks, biological graphs, and social networks, confirming its generalization capability.
• The ablation study confirms that the spectral-based constraint is more effective than feature-only or topology-only constraints in maintaining community invariance.