[Paper Review] Structure-Preserving Graph Representation Learning

Ruiyi Fang, Liangjian Wen+2 more|arXiv (Cornell University)|Sep 2, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes Structure-Preserving Graph Representation Learning (SPGRL), a novel method that enhances graph representation learning by maximizing mutual information (MI) between the original graph topology and feature embeddings. By constructing a k-NN-based feature graph for contrastive learning and enforcing self-reconstruction via MI maximization, SPGRL achieves state-of-the-art performance on semi-supervised node classification and superior robustness under feature and structural noise perturbations.

ABSTRACT

Though graph representation learning (GRL) has made significant progress, it is still a challenge to extract and embed the rich topological structure and feature information in an adequate way. Most existing methods focus on local structure and fail to fully incorporate the global topological structure. To this end, we propose a novel Structure-Preserving Graph Representation Learning (SPGRL) method, to fully capture the structure information of graphs. Specifically, to reduce the uncertainty and misinformation of the original graph, we construct a feature graph as a complementary view via k-Nearest Neighbor method. The feature graph can be used to contrast at node-level to capture the local relation. Besides, we retain the global topological structure information by maximizing the mutual information (MI) of the whole graph and feature embeddings, which is theoretically reduced to exchanging the feature embeddings of the feature and the original graphs to reconstruct themselves. Extensive experiments show that our method has quite superior performance on semi-supervised node classification task and excellent robustness under noise perturbation on graph structure or node features.

Motivation & Objective

• Address the limitation of existing graph representation learning methods in capturing both local and global topological structures.
• Overcome the over-smoothing and feature discrimination degradation caused by deep GCNs in multi-layer settings.
• Reduce reliance on data augmentation and domain-specific engineering by leveraging mutual information maximization.
• Improve robustness to noise in node features and graph structure through structure-preserving representation learning.
• Develop a self-supervised framework that effectively fuses topological and feature information without requiring extensive hyperparameter tuning.

Proposed method

• Construct a k-Nearest Neighbor (k-NN) feature graph as a complementary view to enhance feature propagation and reduce uncertainty in the original graph.
• Use contrastive learning at the node level between the original graph and the feature graph to capture local structural and feature relationships.
• Maximize the mutual information (MI) between the global graph embedding and the global feature embedding to preserve global topological structure.
• Theoretically reformulate MI maximization as a self-reconstruction task, where embeddings from the original and feature graphs are exchanged and reconstructed.
• Train the model end-to-end using a contrastive objective and a mutual information maximization objective to jointly optimize local and global structure preservation.
• Integrate the method within a GCN-based encoder framework to produce low-dimensional, discriminative node representations.

Experimental results

Research questions

• RQ1 Can maximizing mutual information between graph topology and feature representations improve the quality of learned node embeddings?
• RQ2 How does SPGRL perform compared to state-of-the-art methods in semi-supervised node classification under low label rates?
• RQ3 To what extent does SPGRL maintain performance under noise perturbations to node features or graph structure?
• RQ4 How sensitive is SPGRL to hyperparameters such as k in the k-NN construction?
• RQ5 Can a self-reconstruction-based MI maximization strategy effectively preserve global structural information without relying on data augmentation?

Key findings

• SPGRL achieves state-of-the-art performance on semi-supervised node classification, outperforming AMGCN, SCRL, and GCA by an average of 5.87%, 1.91%, and 4.40% respectively on citation networks.
• On the Citeseer dataset with only 0.5% label rate, SPGRL maintains high accuracy (66.2% at 60 label/cluster), significantly outperforming GCN, ChebNet, and GAT, which degrade severely under low supervision.
• Under Gaussian noise perturbation (σ=1.0), SPGRL achieves 73.2% accuracy on ACM at 20 label/cluster, surpassing SCRL (62.8%) and GFNN (52.6%), demonstrating strong robustness.
• The method remains effective across a wide range of k values, with performance peaking at moderate k and declining only slightly at very high k, indicating stability and insensitivity to hyperparameter choice.
• SPGRL maintains competitive performance across multiple datasets (ACM, BlogCatalog, UAI2010, Flickr, Citeseer), showing consistent superiority in both clean and noisy settings.
• The ablation study confirms that MI maximization between topology and feature embeddings is the key component, as it enables the model to preserve global structure and improve generalization.