[Paper Review] Tradeoffs for nearest neighbors on the sphere

Thijs Laarhoven|arXiv (Cornell University)|Nov 24, 2015

Computational Geometry and Mesh Generation17 references12 citations

TL;DR

This paper introduces a novel tradeoff framework for approximate nearest neighbor search on the sphere using spherical filters, optimizing between query and update complexity. It derives a tight tradeoff equation involving the approximation factor $c$, query exponent $ ho_{ ext{q}}$, and update exponent $ ho_{ ext{u}}$, achieving asymptotically optimal performance that matches known lower bounds and improves upon prior LSH-based methods, especially in sparse regimes and for large $c$.

ABSTRACT

We consider tradeoffs between the query and update complexities for the (approximate) nearest neighbor problem on the sphere, extending the recent spherical filters to sparse regimes and generalizing the scheme and analysis to account for different tradeoffs. In a nutshell, for the sparse regime the tradeoff between the query complexity $n^{ρ_q}$ and update complexity $n^{ρ_u}$ for data sets of size $n$ is given by the following equation in terms of the approximation factor $c$ and the exponents $ρ_q$ and $ρ_u$: $$c^2\sqrt{ρ_q}+(c^2-1)\sqrt{ρ_u}=\sqrt{2c^2-1}.$$ For small $c=1+ε$, minimizing the time for updates leads to a linear space complexity at the cost of a query time complexity $n^{1-4ε^2}$. Balancing the query and update costs leads to optimal complexities $n^{1/(2c^2-1)}$, matching bounds from [Andoni-Razenshteyn, 2015] and [Dubiner, IEEE-TIT'10] and matching the asymptotic complexities of [Andoni-Razenshteyn, STOC'15] and [Andoni-Indyk-Laarhoven-Razenshteyn-Schmidt, NIPS'15]. A subpolynomial query time complexity $n^{o(1)}$ can be achieved at the cost of a space complexity of the order $n^{1/(4ε^2)}$, matching the bound $n^{Ω(1/ε^2)}$ of [Andoni-Indyk-Patrascu, FOCS'06] and [Panigrahy-Talwar-Wieder, FOCS'10] and improving upon results of [Indyk-Motwani, STOC'98] and [Kushilevitz-Ostrovsky-Rabani, STOC'98]. For large $c$, minimizing the update complexity results in a query complexity of $n^{2/c^2+O(1/c^4)}$, improving upon the related exponent for large $c$ of [Kapralov, PODS'15] by a factor $2$, and matching the bound $n^{Ω(1/c^2)}$ of [Panigrahy-Talwar-Wieder, FOCS'08]. Balancing the costs leads to optimal complexities $n^{1/(2c^2-1)}$, while a minimum query time complexity can be achieved with update complexity $n^{2/c^2+O(1/c^4)}$, improving upon the previous best exponents of Kapralov by a factor $2$.

Motivation & Objective

• Address the tradeoff between query and update complexity in approximate nearest neighbor search on the sphere, particularly in sparse data regimes.
• Extend spherical filtering techniques to improve upon existing locality-sensitive hashing (LSH) methods in terms of query time and space efficiency.
• Derive a tight analytical tradeoff between query ($n^{ ho_{ ext{q}}}$) and update ($n^{ ho_{ ext{u}}}$) complexities for a given approximation factor $c$.
• Match or improve upon known lower bounds for approximate nearest neighbor search in high-dimensional spaces.
• Explore the feasibility of combining spherical filters with efficient hashing schemes like cross-polytope LSH to reduce decoding overhead.

Proposed method

• Propose a spherical filtering framework that maps data points to spherical caps, enabling efficient filtering of candidates without requiring full partitioning of space.
• Use concatenated codes $C = C_1 \times \cdots \times C_m$ to construct filters with controlled covering radius, ensuring no false negatives in the search process.
• Derive a key tradeoff equation: $ c^{2}\sqrt{\rho_{\text{q}}} + (c^{2}-1)\sqrt{\rho_{\text{u}}} = \sqrt{2c^{2}-1} $, which governs the balance between query and update complexity.
• Analyze the asymptotic behavior of the tradeoff for small and large $c$, showing optimality in both regimes by matching known lower bounds.
• Utilize sparse subcodes (e.g., Leech lattice-based or random sparse codes) to reduce decoding complexity while preserving performance.
• Apply the framework to the random setting on the sphere, where data points are assumed to be distributed with angular distances governed by $c$, and derive performance bounds accordingly.

Experimental results

Research questions

• RQ1 What is the optimal tradeoff between query and update complexity for approximate nearest neighbor search on the sphere using spherical filters?
• RQ2 How does the proposed tradeoff compare to known lower bounds for approximate nearest neighbor search in high-dimensional spaces?
• RQ3 Can the tradeoff be optimized to achieve subpolynomial query time with minimal space overhead, and how does this compare to prior LSH-based methods?
• RQ4 Is the derived tradeoff optimal, and can it be proven to be tight using theoretical bounds or conjectures?
• RQ5 Can spherical filters be combined with efficient hashing schemes like cross-polytope LSH to reduce decoding complexity while maintaining asymptotic optimality?

Key findings

• The proposed tradeoff equation $ c^{2}\sqrt{\rho_{\text{q}}} + (c^{2}-1)\sqrt{\rho_{\text{u}}} = \sqrt{2c^{2}-1} $ characterizes the optimal balance between query and update complexity for spherical approximate nearest neighbor search.
• For small $c = 1 + \varepsilon$, balancing query and update costs yields $n^{1/(2c^2 - 1)}$ complexity, matching the lower bounds from Andoni and Razenshteyn (2015) and Dubiner (2010).
• A subpolynomial query time of $n^{o(1)}$ is achievable at the cost of space complexity $n^{1/(4\varepsilon^2)}$, matching the lower bound $n^{\Omega(1/\varepsilon^2)}$ from Panigrahy et al. (2008, 2010).
• For large $c$, minimizing update complexity leads to query complexity $n^{2/c^2 + O(1/c^4)}$, improving upon Kapralov (2015) by a factor of 2 in the exponent.
• Balancing query and update costs yields optimal complexities of order $n^{1/(2c^2 - 1)}$, matching known lower bounds and improving upon prior LSH schemes.
• The framework achieves asymptotic optimality in both sparse and dense regimes, and the tradeoff is conjectured to be information-theoretically optimal, with a proposed conjecture involving $\sqrt{\rho_{\text{q}}} + \sqrt{\rho_{\text{u}}} \cos\theta \geq \sin\theta$.