[Paper Review] UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

Leland McInnes, John J. Healy+1 more|arXiv (Cornell University)|Feb 9, 2018

Advanced Vision and Imaging43 references7,412 citations

TL;DR

UMAP is a novel manifold learning technique for dimension reduction that is theoretically grounded in Riemannian geometry and algebraic topology, offering competitive visualization quality with t-SNE, better global structure preservation, and scalable performance without embedding-dimension limits.

ABSTRACT

UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for dimension reduction. UMAP is constructed from a theoretical framework based in Riemannian geometry and algebraic topology. The result is a practical scalable algorithm that applies to real world data. The UMAP algorithm is competitive with t-SNE for visualization quality, and arguably preserves more of the global structure with superior run time performance. Furthermore, UMAP has no computational restrictions on embedding dimension, making it viable as a general purpose dimension reduction technique for machine learning.

Motivation & Objective

• Provide a theoretically grounded, scalable method for dimension reduction based on manifold learning.
• Preserve as much of the local and global structure of the data as possible during embedding.
• Develop a practical algorithm that can handle large real-world datasets efficiently.
• Establish a clear connection to Riemannian geometry and topological data analysis to justify design choices.

Proposed method

• Approximate geodesic distances on a manifold by locally normalizing distances to k-nearest neighbors, yielding a per-point local metric.
• Construct fuzzy topological representations (fuzzy simplicial sets) from local metric spaces and fuse them into a global representation via fuzzy union.
• Represent the data manifold as a weighted k-nearest neighbor graph with edge weights reflecting local connectivity and metric scaling.
• Embed data by optimizing a low-dimensional layout to minimize cross-entropy between the high-dimensional and low-dimensional fuzzy topological representations, using a 1-skeleton focus.
• Employ a force-directed graph layout in the embedding space with carefully designed attractive and repulsive forces and a staged annealing-like schedule for stability.
• Base the construction and optimization on mathematical foundations linking simplicial sets, metric spaces, and adjoint functors to ensure principled decisions.

Experimental results

Research questions

• RQ1 How can one construct a faithful topological representation of high-dimensional data to guide dimension reduction?
• RQ2 Can a locally adaptive, geometry-aware graph representation yield scalable and meaningful low-dimensional embeddings?
• RQ3 How does the UMAP embedding quality and runtime compare to established methods like t-SNE and LargeVis on real-world datasets?
• RQ4 What hyper-parameter choices and algorithmic steps are justified by a solid theoretical framework rather than task-specific experimentation?

Key findings

• UMAP achieves competitive visualization quality compared to t-SNE.
• It arguably preserves more of the global data structure than some alternative methods.
• The algorithm demonstrates superior runtime performance and scalability to larger datasets.
• There are no computational restrictions on embedding dimension, enabling UMAP as a general-purpose dimensionality-reduction tool.
• The method is grounded in a rigorous mathematical framework, combining Riemannian geometry and topological data analysis with a practical, scalable implementation.
• UMAP has seen widespread use across fields such as bioinformatics, materials science, and machine learning.