[Paper Review] Hyperbolic Graph Neural Networks

Qi Liu, Maximilian Nickel+1 more|arXiv (Cornell University)|Oct 28, 2019

Image Processing and 3D Reconstruction77 citations

TL;DR

The paper generalizes graph neural networks to Riemannian manifolds and shows hyperbolic GNNs (Poincaré and Lorentz models) outperform Euclidean GNNs on synthetic, molecular, and blockchain graph tasks.

ABSTRACT

Learning from graph-structured data is an important task in machine learning and artificial intelligence, for which Graph Neural Networks (GNNs) have shown great promise. Motivated by recent advances in geometric representation learning, we propose a novel GNN architecture for learning representations on Riemannian manifolds with differentiable exponential and logarithmic maps. We develop a scalable algorithm for modeling the structural properties of graphs, comparing Euclidean and hyperbolic geometry. In our experiments, we show that hyperbolic GNNs can lead to substantial improvements on various benchmark datasets.

Motivation & Objective

• Generalize graph neural networks to operate on arbitrary Riemannian manifolds with differentiable exponential and logarithmic maps.
• Compare Euclidean GNNs with hyperbolic variants on structured graph data.
• Demonstrate improved performance of hyperbolic models for full-graph classification and molecular property prediction.
• Provide scalable, manifold-agnostic framework and release code/data for replication.

Proposed method

• Extend GNN message passing to manifolds by using log maps to map neighborhood features to a tangent space, apply linear transformations, and map back via exp maps.
• Explore Euclidean space, the Poincaré ball, and Lorentz model as hyperbolic manifolds for embeddings.
• Use a centroid-based regression/classification scheme on manifolds to produce graph-level outputs.
• Handle multi-relational data (e.g., ZINC) by incorporating relation-specific weight matrices in message passing.
• Ensure manifold-preserving nonlinearities and discuss activation choices and optimization methods.

Experimental results

Research questions

• RQ1 Can hyperbolic geometry provide a stronger inductive bias for capturing graph structure than Euclidean geometry?
• RQ2 Do hyperbolic graph neural networks improve full-graph classification and molecular property prediction compared to Euclidean GNNs?
• RQ3 Between the Poincaré ball and Lorentz models, which hyperbolic representation yields better performance and stability?
• RQ4 How can outputs be regressed/classified when embeddings live on a manifold rather than Euclidean space?

Key findings

• Hyperbolic GNNs outperform Euclidean GNNs on synthetic structural classification tasks, with larger gains at lower dimensions and Lorentz often outperforming Poincaré.
• On molecular property prediction (ZINC dataset), Lorentz hyperbolic models achieve lower mean absolute error across multiple properties compared to Euclidean and Poincaré models.
• In blockchain transaction graphs, hyperbolic models (Lorentz and Poincaré) yield higher accuracy in predicting price fluctuations than Euclidean baselines.
• Hyperbolic representations effectively capture hierarchical structure in graphs, enabling efficient and accurate modeling across diverse domains.
• The approach remains computationally competitive with Euclidean models, with numerical stability advantages observed for the Lorentz model.