[Paper Review] Unsupervised Attributed Multiplex Network Embedding

Chanyoung Park, Donghyun Kim+2 more|arXiv (Cornell University)|Nov 15, 2019

Advanced Graph Neural NetworksComputer Science40 references20 citations

TL;DR

This paper proposes DMGI, a fully unsupervised method for attributed multiplex network embedding that jointly learns node representations across multiple relation types by leveraging consensus regularization and a universal discriminator inspired by Deep Graph Infomax. DMGI outperforms state-of-the-art methods in node clustering, similarity search, and classification—achieving SOTA results without requiring node labels or explicit supervision on relation types.

ABSTRACT

Nodes in a multiplex network are connected by multiple types of relations. However, most existing network embedding methods assume that only a single type of relation exists between nodes. Even for those that consider the multiplexity of a network, they overlook node attributes, resort to node labels for training, and fail to model the global properties of a graph. We present a simple yet effective unsupervised network embedding method for attributed multiplex network called DMGI, inspired by Deep Graph Infomax (DGI) that maximizes the mutual information between local patches of a graph, and the global representation of the entire graph. We devise a systematic way to jointly integrate the node embeddings from multiple graphs by introducing 1) the consensus regularization framework that minimizes the disagreements among the relation-type specific node embeddings, and 2) the universal discriminator that discriminates true samples regardless of the relation types. We also show that the attention mechanism infers the importance of each relation type, and thus can be useful for filtering unnecessary relation types as a preprocessing step. Extensive experiments on various downstream tasks demonstrate that DMGI outperforms the state-of-the-art methods, even though DMGI is fully unsupervised.

Motivation & Objective

• Address the gap in existing network embedding methods that fail to jointly model multiple relation types and node attributes in multiplex networks.
• Overcome the limitation of prior unsupervised methods that rely on local inductive biases (e.g., GCNs or random walks) and fail to capture global graph structure.
• Develop a framework that enables mutual enhancement among relation-specific embeddings while preserving node attribute information.
• Enable self-supervised representation learning without requiring node labels, making it suitable for real-world networks where labeling is costly.
• Provide interpretability by learning the importance of each relation type via attention, enabling preprocessing for relation filtering.

Proposed method

• Adapt Deep Graph Infomax (DGI) to multiplex networks by training a node encoder to maximize mutual information between local node patches and a global graph-level representation.
• Introduce a consensus regularization framework that minimizes disagreement among relation-type-specific node embeddings, promoting alignment across different graph views.
• Design a universal discriminator that distinguishes true (graph-level summary, local patch) pairs across all relation types, enabling joint training without relation-specific discrimination.
• Employ a shared scoring matrix M to model interactions among multiple relation types, enabling cross-view information flow.
• Use graph convolutional networks (GCNs) to encode node attributes and local structure, integrating them into the global representation via the DGI objective.
• Apply an attention mechanism to learn the importance of each relation type, enabling dynamic weighting and potential filtering of irrelevant relations during preprocessing.

Experimental results

Research questions

• RQ1 Can a fully unsupervised framework jointly learn high-quality node embeddings across multiple relation types in an attributed multiplex network?
• RQ2 How effective is consensus regularization in aligning relation-specific node representations while preserving their distinct structural information?
• RQ3 To what extent does the universal discriminator improve representation learning by unifying supervision across diverse relation types?
• RQ4 Can the attention mechanism in DMGI accurately infer the relative importance of different relation types, and does this aid in preprocessing or relation selection?
• RQ5 Does DMGI achieve state-of-the-art performance on downstream tasks such as node classification, clustering, and similarity search without using node labels?

Key findings

• DMGI achieves the best performance on node classification (F1-macro: 0.778 on DBLP), clustering (NMI: 0.554), and similarity search (Sim@5: 0.798), outperforming SOTA unsupervised methods.
• Ablation studies show that removing the second term in the consensus regularization objective (Eqn. 6) reduces F1-macro to 0.749, indicating its critical role in alignment.
• Using separate scoring matrices for each relation type (M ≠ M^(1) ≠ ... ≠ M^(|R|)) degrades performance significantly (F1-macro: 0.645), proving the benefit of shared interaction modeling.
• Removing node attributes (using adjacency matrix as input) reduces F1-macro to 0.377, demonstrating the essential role of attribute information in representation quality.
• Shuffling the adjacency matrix instead of the attribute matrix causes a sharp drop in performance (F1-macro: 0.364), confirming that structural information is more critical than attribute permutation.
• The attention mechanism in DMGI ${}_{\text{attn}}$ learns meaningful relation weights, with ablation showing that using random sampling for readout reduces performance compared to SAGPool or maxpooling.