[논문 리뷰] Hyperbolic Graph Convolutional Neural Networks

Ines Chami, Rex Ying외 2명|arXiv (Cornell University)|2019. 10. 28.

Advanced Graph Neural Networks참고 문헌 52인용 수 263

한 줄 요약

HGCN은 학습 가능한 곡률을 사용하여 GCN을 하이퍼볼릭 공간으로 확장함으로써 계층적이고 스케일-프리 그래프에서 inductive 학습을 가능하게 하고, 링크 예측 및 노드 분류를 개선한다.

ABSTRACT

Graph convolutional neural networks (GCNs) embed nodes in a graph into Euclidean space, which has been shown to incur a large distortion when embedding real-world graphs with scale-free or hierarchical structure. Hyperbolic geometry offers an exciting alternative, as it enables embeddings with much smaller distortion. However, extending GCNs to hyperbolic geometry presents several unique challenges because it is not clear how to define neural network operations, such as feature transformation and aggregation, in hyperbolic space. Furthermore, since input features are often Euclidean, it is unclear how to transform the features into hyperbolic embeddings with the right amount of curvature. Here we propose Hyperbolic Graph Convolutional Neural Network (HGCN), the first inductive hyperbolic GCN that leverages both the expressiveness of GCNs and hyperbolic geometry to learn inductive node representations for hierarchical and scale-free graphs. We derive GCN operations in the hyperboloid model of hyperbolic space and map Euclidean input features to embeddings in hyperbolic spaces with different trainable curvature at each layer. Experiments demonstrate that HGCN learns embeddings that preserve hierarchical structure, and leads to improved performance when compared to Euclidean analogs, even with very low dimensional embeddings: compared to state-of-the-art GCNs, HGCN achieves an error reduction of up to 63.1% in ROC AUC for link prediction and of up to 47.5% in F1 score for node classification, also improving state-of-the art on the Pubmed dataset.

연구 동기 및 목표

• Motivate the distortion limits of Euclidean GCNs on scale-free and hierarchical graphs.
• Develop inductive hyperbolic graph convolution operations that map Euclidean features into hyperbolic space.
• Propose hyperbolic attention-based aggregation and layer-wise curvature learning for low-distortion embeddings.
• Enable end-to-end training of hyperbolic feature transforms, aggregations, and curvature across layers.
• Demonstrate state-of-the-art performance on link prediction and node classification using real-world hierarchical datasets.

제안 방법

• Derive GCN-style feature transformation and neighborhood aggregation in the hyperboloid model of hyperbolic space.
• Map Euclidean input features to hyperbolic embeddings via the exponential map at a reference north pole.
• Define hyperboloid matrix multiplication using log and exp maps to perform transforms in tangent space.
• Introduce hyperbolic attention-based aggregation operating in local tangent spaces with curvature-aware softmax weighting.
• Implement a non-linear activation across layers with trainable curvatures to transition between hyperbolic spaces.
• Use Fermi-Dirac decoder for link prediction and tangent-space logistic regression for node classification, with end-to-end training of weights, curvatures, and attention.]
• research_questions: ["Can Euclidean input features be effectively transformed into hyperbolic embeddings suitable for graph representation learning?","How can neighborhood aggregation be performed in hyperbolic space to capture hierarchical graph structure?","What is the impact of allowing different trainable curvatures per GCN layer on representation quality and optimization?","Do hyperbolic GCNs improve inductive learning on scale-free and hierarchical graphs for link prediction and node classification?","How do hyperbolic attention mechanisms compare to Euclidean attention in preserving hierarchy in graphs?"]
• key_findings["HGCN significantly improves over Euclidean GNNs on scale-free and hierarchical graphs, with up to 63.1% ROC AUC improvement for link prediction and up to 47.5% F1 score improvement for node classification in certain datasets.","Hyperbolic attention-based aggregation and layer-wise trainable curvature contribute to performance gains over fixed-curvature or no-attention variants.","Training in the hyperboloid model with a stable exponential map yields better optimization than Poincaré-based approaches in their experiments.","HGCN achieves state-of-the-art results on PubMed and demonstrates strong inductive capability across diverse datasets including Disease, Airport, and PPI networks.","Analyses show that HGCN embeddings reflect hierarchical structure and that curvature adaptation helps control embedding scale and distortion."]
• table_headers: []
• table_rows: []}生

실험 결과

연구 질문

• RQ1 Can Euclidean input features be effectively transformed into hyperbolic embeddings suitable for graph representation learning?
• RQ2 How can neighborhood aggregation be performed in hyperbolic space to capture hierarchical graph structure?
• RQ3 What is the impact of allowing different trainable curvatures per GCN layer on representation quality and optimization?
• RQ4 Do hyperbolic GCNs improve inductive learning on scale-free and hierarchical graphs for link prediction and node classification?
• RQ5 How do hyperbolic attention mechanisms compare to Euclidean attention in preserving hierarchy in graphs?

주요 결과

• HGCN significantly improves over Euclidean GNNs on scale-free and hierarchical graphs, with up to 63.1% ROC AUC improvement for link prediction and up to 47.5% F1 score improvement for node classification in certain datasets.
• Hyperbolic attention-based aggregation and layer-wise trainable curvature contribute to performance gains over fixed-curvature or no-attention variants.
• Training in the hyperboloid model with a stable exponential map yields better optimization than Poincaré-based approaches in their experiments.
• HGCN achieves state-of-the-art results on PubMed and demonstrates strong inductive capability across diverse datasets including Disease, Airport, and PPI networks.
• Analyses show that HGCN embeddings reflect hierarchical structure and that curvature adaptation helps control embedding scale and distortion.