[Paper Review] Heterogeneous Deep Graph Infomax

Yuxiang Ren, Бо Лю+4 more|arXiv (Cornell University)|Nov 19, 2019

Advanced Graph Neural Networks44 references66 citations

TL;DR

HDGI is an unsupervised graph neural network for heterogeneous graphs that maximizes local-global mutual information across meta-path induced semantics, using semantic-level attention to fuse multiple meta-paths. It achieves state-of-the-art results on node classification and clustering without supervision.

ABSTRACT

Graph representation learning is to learn universal node representations that preserve both node attributes and structural information. The derived node representations can be used to serve various downstream tasks, such as node classification and node clustering. When a graph is heterogeneous, the problem becomes more challenging than the homogeneous graph node learning problem. Inspired by the emerging information theoretic-based learning algorithm, in this paper we propose an unsupervised graph neural network Heterogeneous Deep Graph Infomax (HDGI) for heterogeneous graph representation learning. We use the meta-path structure to analyze the connections involving semantics in heterogeneous graphs and utilize graph convolution module and semantic-level attention mechanism to capture local representations. By maximizing local-global mutual information, HDGI effectively learns high-level node representations that can be utilized in downstream graph-related tasks. Experiment results show that HDGI remarkably outperforms state-of-the-art unsupervised graph representation learning methods on both classification and clustering tasks. By feeding the learned representations into a parametric model, such as logistic regression, we even achieve comparable performance in node classification tasks when comparing with state-of-the-art supervised end-to-end GNN models.

Motivation & Objective

• Motivate and address unsupervised representation learning on heterogeneous graphs.
• Model how multi-typed nodes/edges convey rich semantics through meta-paths.
• Propose an MI-based objective to learn root representations without labels.
• Leverage meta-path specific encoders and semantic-level attention to fuse semantics.
• Demonstrate effectiveness on node classification and clustering tasks against baselines.

Proposed method

• Define heterogeneous graphs and meta-path based adjacency matrices for multiple semantics.
• Compute meta-path specific node representations with GCN or GAT on each homogeneous subgraph.
• Aggregate semantics with a semantic-level attention to obtain a joint node representation H.
• Derive a graph-level summary vector s with a global encoder (averaging, pooling, or Set2vec).
• Maximize mutual information between local node representations H and global summary s using a discriminator D with negative sampling.
• Generate negative samples by shuffling node features while keeping meta-path adjacencies fixed to form Neg pairs; train with a binary cross-entropy loss that lower-bounds MI.
• Provide end-to-end training via backpropagation to learn representations without labels.

Experimental results

Research questions

• RQ1 Can MI-based objectives be effectively extended from homogeneous to heterogeneous graphs?
• RQ2 Do meta-paths plus semantic-level attention capture diverse semantics in HGs for robust representations?
• RQ3 How does HDGI perform on unsupervised node classification and clustering compared to supervised GNNs and other unsupervised methods?
• RQ4 What is the impact of different global encoders (averaging, pooling, Set2vec) on learned representations?
• RQ5 Does negative sampling quality affect mutual-information maximization in heterogeneous settings?

Key findings

• HDGI outperforms state-of-the-art unsupervised methods on node classification and clustering across multiple heterogeneous datasets.
• HDGI-C and HDGI-A achieve strong results, often surpassing supervised and HAN baselines on node classification.
• Meta-path based attention effectively integrates semantics from PAP, PSP, MAM, MDM, MKM, etc., improving representation quality.
• The MI-based objective with a learned discriminator encourages representations that preserve global graph information while incorporating local attributes.
• HDGI's unsupervised representations are competitive with or superior to end-to-end supervised GNN models when used with simple downstream classifiers.
• Experiments on ACM, DBLP, and IMDB datasets show HDGI's robustness across different HG structures and metadata.