[Paper Review] Hop-Hop Relation-aware Graph Neural Networks

Li Zhang, Yan Ge+1 more|arXiv (Cornell University)|Dec 21, 2020

Advanced Graph Neural Networks45 references4 citations

TL;DR

This paper proposes Hop-Hop Relation-aware Graph Neural Networks (HHR-GNN), a unified framework for homogeneous and heterogeneous graph representation learning that uses knowledge graph embedding to learn personalized, hop-aware receptive fields via relation-scores. By applying hop-aware projection and aggregation using learned relation-scores, HHR-GNN achieves state-of-the-art accuracy and up to 13Kx faster training per epoch on large heterogeneous graphs compared to prior methods.

ABSTRACT

Graph Neural Networks (GNNs) are widely used in graph representation learning. However, most GNN methods are designed for either homogeneous or heterogeneous graphs. In this paper, we propose a new model, Hop-Hop Relation-aware Graph Neural Network (HHR-GNN), to unify representation learning for these two types of graphs. HHR-GNN learns a personalized receptive field for each node by leveraging knowledge graph embedding to learn relation scores between the central node's representations at different hops. In neighborhood aggregation, our model simultaneously allows for hop-aware projection and aggregation. This mechanism enables the central node to learn a hop-wise neighborhood mixing that can be applied to both homogeneous and heterogeneous graphs. Experimental results on five benchmarks show the competitive performance of our model compared to state-of-the-art GNNs, e.g., up to 13K faster in terms of time cost per training epoch on large heterogeneous graphs.

Motivation & Objective

• Address the limitations of fixed receptive fields in existing GNNs, which fail to adapt to varying node connectivity patterns in both homogeneous and heterogeneous graphs.
• Overcome the issue of treating all neighbors equally in neighborhood aggregation, especially in heterogeneous graphs where different hops and node types have distinct relevance to the central node.
• Enable efficient and interpretable representation learning by learning dynamic, personalized receptive fields using knowledge graph embedding techniques.
• Unify representation learning across homogeneous and heterogeneous graphs under a single framework that supports hop-aware projection and aggregation.
• Improve model efficiency and interpretability by leveraging lightweight neural tensor networks to compute relation-scores between central nodes and multi-hop neighbors.

Proposed method

• Introduce a novel GNN architecture that combines graph neural networks with knowledge graph embedding (KGE) to compute relation-scores between a central node and its multi-hop neighbors.
• Use a neural tensor network (NTN) to compute relation-scores (e.g., α₀₁, α₀₂) between the central node and its representations at different hops, enabling soft, personalized aggregation.
• Apply hop-aware projection by using distinct weight matrices for each hop’s neighbor embeddings, allowing different feature space mappings based on hop distance.
• Perform weighted aggregation of multi-hop neighbor representations using learned relation-scores, followed by concatenation with the central node’s embedding to form the final representation.
• Design a unified framework where the same architecture handles both homogeneous and heterogeneous graphs by treating different node types and hop distances as distinct relation types.
• Leverage low-dimensional GNN embeddings as input to the NTN module to ensure computational efficiency, especially in large-scale graphs with limited node types.

Experimental results

Research questions

• RQ1 Can a unified GNN framework effectively handle both homogeneous and heterogeneous graphs by learning personalized receptive fields per node?
• RQ2 How can hop-aware projection and aggregation improve representation learning when different hops of neighbors have varying relevance to the central node?
• RQ3 To what extent can learned relation-scores from knowledge graph embedding enhance model interpretability and performance in node classification?
• RQ4 Does the proposed method achieve better efficiency than existing GNNs on large-scale heterogeneous graphs, particularly those with complex neighborhood structures?
• RQ5 Can the model dynamically identify which hops or node types contribute most to a node’s representation, enabling interpretable predictions?

Key findings

• HHR-GNN achieves competitive performance on node classification, outperforming state-of-the-art models like GTN and HAN by up to 9.3% on the IMDB dataset.
• On large heterogeneous graphs, HHR-GNN reduces training time per epoch by up to 13,000× compared to existing methods, demonstrating exceptional efficiency.
• The model learns personalized receptive fields through relation-scores, with one-hop neighbors consistently receiving higher scores than two-hop neighbors, confirming their stronger relevance.
• Interpretability is enhanced: visualization of learned relation-scores shows that nodes develop distinct preferences for certain neighbor types (e.g., Movie vs. Director in IMDB), reflecting real-world relationships.
• The use of lightweight NTN-based relation-score learning enables efficient computation, especially in graphs with few node types, while maintaining high accuracy.
• HHR-GNN outperforms GCN in accuracy on heterogeneous graphs, showing that simple neighbor aggregation is insufficient for complex graph structures.