[Paper Review] Tree Structure-Aware Graph Representation Learning via Integrated Hierarchical Aggregation and Relational Metric Learning

Ziyue Qiao, Pengyang Wang+4 more|arXiv (Cornell University)|Aug 23, 2020

Advanced Graph Neural Networks42 references4 citations

TL;DR

This paper proposes T-GNN, a novel graph neural network that jointly preserves hierarchical tree structures and node type heterogeneity in heterogeneous graphs by integrating gated recurrent units for multi-hop neighborhood aggregation and relational metric learning to embed different node types into distinct feature spaces. The method achieves state-of-the-art performance in node classification, clustering, and link prediction tasks.

ABSTRACT

While Graph Neural Network (GNN) has shown superiority in learning node representations of homogeneous graphs, leveraging GNN on heterogeneous graphs remains a challenging problem. The dominating reason is that GNN learns node representations by aggregating neighbors' information regardless of node types. Some work is proposed to alleviate such issue by exploiting relations or meta-path to sample neighbors with distinct categories, then use attention mechanism to learn different importance for different categories. However, one limitation is that the learned representations for different types of nodes should own different feature spaces, while all the above work still project node representations into one feature space. Moreover, after exploring massive heterogeneous graphs, we identify a fact that multiple nodes with the same type always connect to a node with another type, which reveals the many-to-one schema, a.k.a. the hierarchical tree structure. But all the above work cannot preserve such tree structure, since the exact multi-hop path correlation from neighbors to the target node would be erased through aggregation. Therefore, to overcome the limitations of the literature, we propose T-GNN, a tree structure-aware graph neural network model for graph representation learning. Specifically, the proposed T-GNN consists of two modules: (1) the integrated hierarchical aggregation module and (2) the relational metric learning module. The integrated hierarchical aggregation module aims to preserve the tree structure by combining GNN with Gated Recurrent Unit to integrate the hierarchical and sequential neighborhood information on the tree structure to node representations. The relational metric learning module aims to preserve the heterogeneity by embedding each type of nodes into a type-specific space with distinct distribution based on similarity metrics.

Motivation & Objective

• To address the limitation of existing GNNs in preserving multi-hop hierarchical tree structures common in heterogeneous graphs.
• To overcome the issue of shared feature space across different node types, which hinders effective representation learning.
• To model the many-to-one schema (e.g., multiple papers linked to one author) as a tree-structured neighborhood for richer structural inductive bias.
• To improve node representation quality for downstream tasks such as classification, clustering, and link prediction in heterogeneous graphs.

Proposed method

• Integrates gated recurrent units (GRUs) with graph convolutional aggregation to preserve sequential and hierarchical neighborhood information across multi-hop paths.
• Designs an integrated hierarchical aggregation module that captures the order and depth of neighbor nodes in tree-structured neighborhoods.
• Proposes a relational metric learning module that embeds each node type into a type-specific feature space with distinct distribution and similarity metrics.
• Uses relation-specific metric spaces to compute node similarities, reducing interference between different node types during representation learning.
• Combines hierarchical aggregation with metric learning in an end-to-end trainable framework to jointly optimize structural and heterogeneity-aware representations.
• Employs meta-path-inspired neighborhood sampling to define tree-structured multi-hop neighborhoods for each target node.

Experimental results

Research questions

• RQ1 Can a GNN model effectively preserve the hierarchical tree structure inherent in multi-hop neighborhoods of heterogeneous graphs?
• RQ2 Does embedding different node types into type-specific feature spaces improve representation quality compared to shared feature spaces?
• RQ3 Can joint optimization of hierarchical aggregation and relational metric learning lead to better performance in node-level graph learning tasks?
• RQ4 How does T-GNN compare to existing GNNs in handling long-range dependencies and structural heterogeneity in heterogeneous graphs?

Key findings

• T-GNN achieves state-of-the-art performance on node classification and clustering tasks across multiple heterogeneous graph benchmarks, outperforming existing methods such as GAT, HAN, and RGCN.
• The model significantly improves link prediction performance, demonstrating its ability to capture complex structural patterns in heterogeneous graphs.
• Ablation studies confirm that both the hierarchical aggregation and relational metric learning modules contribute independently and synergistically to performance gains.
• Visualization results show that nodes of the same type are embedded into distinct, well-separated spaces with high intra-class similarity and inter-class separation.
• The model effectively preserves multi-hop path correlations, as evidenced by the learned representations capturing the hierarchical structure of neighborhoods like paper-author-organization trees.