[Paper Review] Multi-Label Graph Convolutional Network Representation Learning

Min Shi, Yufei Tang+2 more|arXiv (Cornell University)|Dec 26, 2019

Text and Document Classification Technologies29 references4 citations

TL;DR

This paper proposes Multi-Label Graph Convolutional Network (ML-GCN), a siamese GCN framework that jointly learns node and label representations by modeling both node-node-label and label-label-node graphs. By integrating label correlations and network topology under a unified objective, ML-GCN significantly improves multi-label node classification performance over baseline methods, demonstrating the critical role of label interactions in representation learning.

ABSTRACT

Knowledge representation of graph-based systems is fundamental across many disciplines. To date, most existing methods for representation learning primarily focus on networks with simplex labels, yet real-world objects (nodes) are inherently complex in nature and often contain rich semantics or labels, e.g., a user may belong to diverse interest groups of a social network, resulting in multi-label networks for many applications. The multi-label network nodes not only have multiple labels for each node, such labels are often highly correlated making existing methods ineffective or fail to handle such correlation for node representation learning. In this paper, we propose a novel multi-label graph convolutional network (ML-GCN) for learning node representation for multi-label networks. To fully explore label-label correlation and network topology structures, we propose to model a multi-label network as two Siamese GCNs: a node-node-label graph and a label-label-node graph. The two GCNs each handle one aspect of representation learning for nodes and labels, respectively, and they are seamlessly integrated under one objective function. The learned label representations can effectively preserve the inner-label interaction and node label properties, and are then aggregated to enhance the node representation learning under a unified training framework. Experiments and comparisons on multi-label node classification validate the effectiveness of our proposed approach.

Motivation & Objective

• To address the limitation of existing graph representation learning methods that focus only on simplex labels, which fail to capture complex, correlated multi-label semantics in real-world networks.
• To model the mutual reinforcement between node representations and label representations by exploiting label-label correlations and network topology simultaneously.
• To develop a unified training framework that integrates node and label representation learning under a single objective function.
• To validate that incorporating label correlations enhances node representation learning, especially for under-represented or correlated labels.
• To demonstrate the superiority of the proposed ML-GCN model over existing GCN-based and multi-label learning baselines on real-world multi-label networks.

Proposed method

• The method constructs two siamese GCNs: a node-node-label graph for learning node representations and a label-label-node graph for learning label representations.
• The label-label-node graph models label correlations by treating labels as nodes connected through shared labeled nodes, capturing inter-label dependencies.
• Node and label representations are learned jointly via a unified optimization objective, enabling mutual refinement during training.
• The model uses graph convolutional layers to aggregate features from neighboring nodes and correlated labels, with learnable parameters updated end-to-end.
• Feature updating frequencies (N and M) are introduced to control the synchronization of node and label representation updates during training.
• The final node representation is enhanced by aggregating label representations that encode label correlations and shared community structures.

Experimental results

Research questions

• RQ1 Can joint learning of node and label representations improve multi-label node classification beyond traditional GCNs?
• RQ2 How do label correlations influence the quality of node representation learning in multi-label networks?
• RQ3 What is the impact of modeling label-label interactions via a separate label-label-node graph on overall performance?
• RQ4 How do different architectural choices (e.g., GCN depth, feature update frequency) affect the model’s ability to capture label correlations and topology?
• RQ5 To what extent does incorporating label correlation improve performance on low-frequency or highly correlated labels?

Key findings

• ML-GCN outperforms GCN and other variants like ML-GCN node, ML-GCN 1l, and ML-GCN 2l across all datasets, demonstrating the effectiveness of joint node-label learning.
• The model achieves higher Micro-F1 and Macro-F1 scores as the number of labeled training nodes increases, confirming its reliance on supervised signal.
• Performance peaks at an embedding size of 400–450 for the first convolutional layer, after which accuracy sharply declines, indicating optimal dimensionality.
• Label correlation significantly enhances classification performance: categories with stronger correlations to a high-impact label (e.g., L1) show greater improvement in accuracy when using ML-GCN.
• The case study confirms that label interactions are critical—ML-GCN improves performance on L2, L3, and L4, which are correlated with L1, while GCN does not.
• The ablation study shows that using common nodes as attributes in the label-label network improves label representation learning, which in turn enhances node representation aggregation.