[Paper Review] Tree! I am no Tree! I am a Low Dimensional Hyperbolic Embedding

Rishi Sonthalia, Anna C. Gilbert|arXiv (Cornell University)|May 8, 2020

Topological and Geometric Data Analysis33 references5 citations

TL;DR

This paper proposes TreeRep, a fast metric-first algorithm that learns a tree structure from a $δ$-hyperbolic metric to enable low-dimensional hyperbolic embeddings. By constructing a tree that approximates the original metric—exact when $δ=0$—TreeRep achieves over 10,000x speedup over existing hyperbolic embedding methods while delivering lower average distortion and higher mean average precision, enabling scalable hierarchical analysis of large datasets in genomics, linguistics, and social networks.

ABSTRACT

Given data, finding a faithful low-dimensional hyperbolic embedding of the data is a key method by which we can extract hierarchical information or learn representative geometric features of the data. In this paper, we explore a new method for learning hyperbolic representations by taking a metric-first approach. Rather than determining the low-dimensional hyperbolic embedding directly, we learn a tree structure on the data. This tree structure can then be used directly to extract hierarchical information, embedded into a hyperbolic manifold using Sarkar's construction \cite{sarkar}, or used as a tree approximation of the original metric. To this end, we present a novel fast algorithm extsc{TreeRep} such that, given a $δ$-hyperbolic metric (for any $δ\geq 0$), the algorithm learns a tree structure that approximates the original metric. In the case when $δ= 0$, we show analytically that extsc{TreeRep} exactly recovers the original tree structure. We show empirically that extsc{TreeRep} is not only many orders of magnitude faster than previously known algorithms, but also produces metrics with lower average distortion and higher mean average precision than most previous algorithms for learning hyperbolic embeddings, extracting hierarchical information, and approximating metrics via tree metrics.

Motivation & Objective

• To develop a scalable, efficient method for learning low-dimensional hyperbolic embeddings that preserves hierarchical structure in data.
• To address the lack of geometric guarantees and high computational cost in existing non-convex optimization-based hyperbolic embedding methods.
• To demonstrate that learning a tree structure first—before embedding into hyperbolic space—leads to better metric fidelity and performance.
• To provide a fast, analytically grounded algorithm that approximates $δ$-hyperbolic metrics with tree metrics, especially exact for $δ=0$.
• To enable hierarchical analysis on large-scale datasets (e.g., single-cell genomics, social networks) previously infeasible due to computational constraints.

Proposed method

• Adopt a metric-first approach: instead of optimizing hyperbolic embeddings directly, learn a tree structure that approximates the input $δ$-hyperbolic metric.
• Use geometric insights from the metric to recursively place Steiner nodes (internal tree nodes) based on distance triplets, leveraging the four-point condition for $δ$-hyperbolicity.
• Apply two recursive zones: Zone 1 places nodes directly connected to a Steiner node based on maximum distance to a pair, while Zone 2 handles nodes requiring edge contraction via recursive refinement.
• Construct the tree by iteratively selecting three random points, computing their pairwise distances, and using the Gromov product to determine the optimal placement of new Steiner nodes.
• Use Sarkar’s construction to embed the resulting tree into hyperbolic space, preserving the hierarchical structure and enabling downstream analysis.
• The algorithm is deterministic and runs in near-linear time, with complexity dependent on the number of points and the metric’s hyperbolicity parameter $δ$.

Experimental results

Research questions

• RQ1 Can a metric-first approach—learning a tree structure before hyperbolic embedding—yield better or more efficient hyperbolic representations than direct optimization?
• RQ2 How accurately can a tree structure approximate a $δ$-hyperbolic metric, especially when $δ=0$?
• RQ3 What is the computational performance gain of TreeRep compared to existing tree approximation and hyperbolic embedding algorithms?
• RQ4 Does TreeRep outperform state-of-the-art methods in terms of average distortion and mean average precision across diverse datasets?
• RQ5 Can TreeRep scale to large-scale datasets such as Enron (180,810 nodes) and WordNet (75,834 nodes) where prior methods fail due to runtime constraints?

Key findings

• TreeRep achieves exact recovery of the original tree structure when $δ=0$, proving analytical correctness for tree-approximate metrics.
• For $δ>0$, TreeRep produces tree metrics with significantly lower average distortion (e.g., 0.131 on Wordnet) than competing methods like Saitou and Nei (0.336) and Abraham et al. (0.607 on Enron).
• TreeRep is over 10,000 times faster than optimization-based hyperbolic embedding methods (e.g., Nickel and Kiela, Sala et al.), with runtime under 1 second on datasets like Celegan (2024 nodes) and under 10 seconds on GRQC (13,422 nodes).
• On the Enron dataset (180,810 nodes), TreeRep runs in 27 seconds, while optimization-based methods are infeasible due to excessive runtime.
• TreeRep achieves higher mean average precision than most prior methods, with 0.380 on CS Phd and 0.332 on Dieseasome, outperforming Saitou and Nei and matching or exceeding other tree approximation algorithms.
• The algorithm scales efficiently: on Grid-worm (6,421 nodes), TreeRep runs in 0.731 seconds, while other methods fail to complete or are prohibitively slow.