[Paper Review] A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction

Guillaume Huguet, Alexander Tong+5 more|arXiv (Cornell University)|May 30, 2023

Cancer-related molecular mechanisms research4 citations

TL;DR

This paper introduces Heat Geodesic Embeddings, a novel dimensionality reduction method that leverages Varadhan's formula to directly link heat diffusion to geodesic distances on manifolds. By formulating a heat-kernel-based dissimilarity measure and using multidimensional scaling, the method preserves intrinsic manifold structure more faithfully than state-of-the-art techniques like PHATE and t-SNE, especially in noisy, high-dimensional data such as single-cell RNA-seq, enabling accurate interpolation of temporal trajectories.

ABSTRACT

Diffusion-based manifold learning methods have proven useful in representation learning and dimensionality reduction of modern high dimensional, high throughput, noisy datasets. Such datasets are especially present in fields like biology and physics. While it is thought that these methods preserve underlying manifold structure of data by learning a proxy for geodesic distances, no specific theoretical links have been established. Here, we establish such a link via results in Riemannian geometry explicitly connecting heat diffusion to manifold distances. In this process, we also formulate a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. This novel perspective makes clearer the choices available in manifold learning and denoising. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances, and preserving cluster structure in toy datasets. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure, where our method enables interpolation of withheld timepoints of data. Finally, we show that parameters of our more general method can be configured to give results similar to PHATE (a state-of-the-art diffusion based manifold learning method) as well as SNE (an attraction/repulsion neighborhood based method that forms the basis of t-SNE).

Motivation & Objective

• To establish a theoretical link between heat diffusion and geodesic distances on Riemannian manifolds.
• To develop a new dimensionality reduction method that explicitly preserves geodesic distances by leveraging heat kernel theory.
• To provide a unified framework that explains and generalizes existing diffusion-based methods like PHATE and t-SNE in terms of geodesic preservation and denoising.
• To enable accurate interpolation of missing timepoints in single-cell RNA-seq data by preserving the underlying manifold structure.
• To offer a theoretically grounded, versatile method with configurable parameters for balancing local vs. global structure and noise robustness.

Proposed method

• Proposes a heat-geodesic dissimilarity measure based on Varadhan’s formula, which connects the logarithm of the heat kernel to squared geodesic distance in the small-time limit.
• Uses the heat kernel to construct a distance matrix that approximates the true geodesic distances on the underlying manifold.
• Applies multidimensional scaling (MDS) to embed the data into a low-dimensional space while preserving the heat-geodesic dissimilarity.
• Incorporates Harnack regularization and efficient heat kernel approximations (Euler and Chebyshev) to improve numerical stability and scalability.
• Configures hyperparameters such as diffusion time, number of neighbors, and approximation order to control the balance between local and global structure.
• Demonstrates that the method can be tuned to produce results similar to PHATE (diffusion-based) or t-SNE (attractive/repulsive force-based) by adjusting the dissimilarity function and loss.

Experimental results

Research questions

• RQ1 Can heat diffusion be theoretically linked to geodesic distances on Riemannian manifolds, and if so, how can this be used to improve dimensionality reduction?
• RQ2 How does the proposed heat geodesic embedding method compare to existing methods like PHATE and t-SNE in preserving true manifold distances?
• RQ3 To what extent can the method interpolate missing data points, such as withheld timepoints in single-cell RNA-seq, by preserving the manifold structure?
• RQ4 How do key hyperparameters—diffusion time, number of neighbors, Harnack regularization, and approximation order—affect the quality of the embedding?
• RQ5 Can the method be configured to reproduce the behavior of existing state-of-the-art methods like PHATE and t-SNE through parameter tuning?

Key findings

• Heat Geodesic Embeddings outperform state-of-the-art methods in preserving ground-truth geodesic distances, achieving a mean correlation of 0.605±0.093 on the Swiss roll dataset with high noise and diffusion time 0.5.
• On the tree dataset, Heat Geo achieves a correlation of 0.784±0.051 with ground truth distances, significantly outperforming PHATE (0.550±0.042) and t-SNE (0.706±0.054).
• The method enables accurate interpolation of withheld timepoints in single-cell RNA-seq data by preserving the manifold structure, demonstrating its utility in developmental biology applications.
• Harnack regularization improves stability and correlation with ground truth distances, particularly in high-noise settings, as shown by increased correlation in ablation studies.
• The method can be tuned to produce results similar to PHATE and t-SNE by adjusting the heat kernel dissimilarity and loss function, demonstrating its theoretical unification of existing approaches.
• Results are robust across different graph construction methods (kNN, UMAP, Scanpy), with consistent performance on the 10-dimensional Swiss roll dataset, indicating generalizability.