[Paper Review] Dimensionality Reduction has Quantifiable Imperfections: Two Geometric Bounds

Kry Yik Chau Lui, Gavin Weiguang Ding+2 more|arXiv (Cornell University)|Oct 31, 2018

Topological and Geometric Data Analysis7 references4 citations

TL;DR

This paper establishes fundamental geometric limits in dimensionality reduction (DR) for information retrieval by proving that no continuous DR map can achieve both perfect precision and perfect recall simultaneously. It introduces a novel Wasserstein-based performance measure with theoretical lower bounds and derives an exponential decay bound for precision under Lipschitz continuity, revealing an inherent trade-off between continuity and injectivity in DR maps.

ABSTRACT

In this paper, we investigate Dimensionality reduction (DR) maps in an information retrieval setting from a quantitative topology point of view. In particular, we show that no DR maps can achieve perfect precision and perfect recall simultaneously. Thus a continuous DR map must have imperfect precision. We further prove an upper bound on the precision of Lipschitz continuous DR maps. While precision is a natural measure in an information retrieval setting, it does not measure `how' wrong the retrieved data is. We therefore propose a new measure based on Wasserstein distance that comes with similar theoretical guarantee. A key technical step in our proofs is a particular optimization problem of the $L_2$-Wasserstein distance over a constrained set of distributions. We provide a complete solution to this optimization problem, which can be of independent interest on the technical side.

Motivation & Objective

• To theoretically analyze the fundamental limitations of dimensionality reduction (DR) maps in information retrieval settings.
• To identify why precision and recall cannot be simultaneously maximized in continuous DR maps.
• To develop a new performance measure based on L2-Wasserstein distance that quantifies geometric distortion and supports theoretical guarantees.
• To establish a rigorous upper bound on precision decay for Lipschitz continuous DR maps.
• To provide a theoretically grounded alternative to heuristic metrics like f-1 score in DR evaluation.

Proposed method

• The authors define continuous analogues of precision and recall in a geometric setting, modeling relevance via neighborhoods in input and feature spaces.
• They prove that perfect recall is equivalent to continuity of the DR map, and that perfect precision and recall cannot coexist due to topological constraints.
• A key technical contribution is solving an L2-Wasserstein distance optimization problem over a constrained set of probability distributions, which underpins the theoretical analysis.
• They derive an exponential upper bound on precision decay as a function of the number of dimensions reduced, under Lipschitz continuity.
• They propose a new performance metric, the $W_2$ measure, based on $L_2$-Wasserstein distance that captures both distance distortion and set mismatch.
• They validate the utility of the Wasserstein measure by showing its lower bound correlates with optimizing the f-β score in practice.

Experimental results

Research questions

• RQ1 Can a continuous dimensionality reduction map achieve both perfect precision and perfect recall simultaneously in an information retrieval setting?
• RQ2 What is the theoretical decay rate of precision for Lipschitz continuous DR maps as dimensionality is reduced?
• RQ3 How can a performance metric be constructed that quantifies geometric distortion beyond precision and recall?
• RQ4 What is the theoretical lower bound for the proposed Wasserstein-based performance measure in continuous DR maps?
• RQ5 Does optimizing the Wasserstein measure lower bound correspond to improved f-β score in practical DR evaluation?

Key findings

• No continuous DR map can achieve both perfect precision and perfect recall simultaneously, establishing a fundamental theoretical trade-off.
• The precision of any Lipschitz continuous DR map decays exponentially with respect to the number of dimensions reduced.
• The proposed $W_2$ measure based on $L_2$-Wasserstein distance provides a theoretically grounded alternative that captures both distance distortion and set mismatch.
• The $W_2$ measure enjoys a theoretical lower bound for continuous DR maps, enabling principled evaluation.
• Simulation results show that optimizing the Wasserstein measure lower bound correlates with improved f-β score, validating its practical relevance.
• Empirical analysis on S-curve, Swiss roll, and MNIST datasets confirms that t-SNE's performance degrades outside the perplexity range of 32–128, aligning with the theoretical trade-off between continuity and injectivity.