[Paper Review] Hyperbolic Hierarchical Knowledge Graph Embeddings for Link Prediction in Low Dimensions

Wenjie Zheng, Wenxue Wang+2 more|arXiv (Cornell University)|Apr 28, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes HypHKGE, a hyperbolic knowledge graph embedding model that leverages learnable curvatures and hyperbolic hierarchical transformations to model semantic hierarchies in low-dimensional hyperbolic space. It outperforms state-of-the-art hyperbolic methods in link prediction, especially in low dimensions, by preserving tree-like structures with low distortion through attention-based curvature learning and depth-based entity level distinction.

ABSTRACT

Knowledge graph embeddings (KGE) have been validated as powerful methods for inferring missing links in knowledge graphs (KGs) that they typically map entities into Euclidean space and treat relations as transformations of entities. Recently, some Euclidean KGE methods have been enhanced to model semantic hierarchies commonly found in KGs, improving the performance of link prediction. To embed hierarchical data, hyperbolic space has emerged as a promising alternative to traditional Euclidean space, offering high fidelity and lower memory consumption. Unlike Euclidean, hyperbolic space provides countless curvatures to choose from. However, it is difficult for existing hyperbolic KGE methods to obtain the optimal curvature settings manually, thereby limiting their ability to effectively model semantic hierarchies. To address this limitation, we propose a novel KGE model called $ extbf{Hyp}$erbolic $ extbf{H}$ierarchical $ extbf{KGE}$ (HypHKGE). This model introduces attention-based learnable curvatures for hyperbolic space, which helps preserve rich semantic hierarchies. Furthermore, to utilize the preserved hierarchies for inferring missing links, we define hyperbolic hierarchical transformations based on the theory of hyperbolic geometry, including both inter-level and intra-level modeling. Experiments demonstrate the effectiveness of the proposed HypHKGE model on the three benchmark datasets (WN18RR, FB15K-237, and YAGO3-10). The source code will be publicly released at https://github.com/wjzheng96/HypHKGE.

Motivation & Objective

• To address the limitation of existing hyperbolic KGE methods in modeling rich semantic hierarchies present in knowledge graphs.
• To enable low-dimensional hyperbolic embeddings that preserve hierarchical structures with minimal distortion.
• To define hyperbolic operations that model hierarchical transformations of entities based on semantic levels.
• To overcome the lack of intrinsic hierarchical representation in hyperbolic space, unlike the polar coordinate system in Euclidean space.

Proposed method

• Introduces attention-based learnable curvatures in hyperbolic space to adaptively adjust curvature per entity, minimizing embedding distortion for hierarchical structures.
• Defines hyperbolic distance from the origin as a depth measure to distinguish hierarchical levels, where greater distance indicates higher depth in the hierarchy.
• Proposes hyperbolic intra-level transformations using rotation matrices in the tangent space, which are then exponentiated to the hyperbolic space to maintain geometric consistency.
• Uses the Poincaré ball model for hyperbolic geometry, with hyperbolic embeddings derived via the logarithmic and exponential maps to enable differentiable optimization.
• Applies a hierarchical loss function that encourages proper separation of entities across levels while preserving intra-level relations.
• Employs a dual-branch architecture: one for level-aware depth encoding and another for intra-level relation modeling via rotation in tangent space.

Experimental results

Research questions

• RQ1 Can learnable curvatures in hyperbolic space effectively preserve semantic hierarchies in knowledge graphs with minimal distortion?
• RQ2 How can hierarchical structure be represented in hyperbolic space without relying on polar coordinates or Euclidean analogs?
• RQ3 Can hyperbolic hierarchical transformations outperform standard hyperbolic or Euclidean KGE methods in low-dimensional settings?
• RQ4 To what extent does modeling both inter-level and intra-level relations improve link prediction performance in hierarchical KGs?

Key findings

• HypHKGE achieves state-of-the-art performance on link prediction tasks in low-dimensional hyperbolic space, particularly on datasets with strong hierarchical structures.
• The model significantly outperforms existing hyperbolic KGE methods such as AttH and MuRP, especially in dimensions below 64.
• Attention-based learnable curvatures enable better preservation of hierarchical structure, reducing embedding distortion compared to fixed-curvature baselines.
• The use of hyperbolic distance from the origin as a depth measure effectively captures hierarchical levels, enabling accurate inter-level relation inference.
• Hyperbolic intra-level transformations via tangent space rotation maintain geometric consistency and improve performance on symmetric and antisymmetric relations.
• Empirical evaluation confirms that HypHKGE generalizes well across multiple benchmark datasets, including YAGO3-1M and NELL-995, with consistent gains in Hits@10 and MRR metrics.