[Paper Review] Reverse Nearest Neighbors Search in High Dimensions using Locality-Sensitive Hashing

David Arthur, Steve Oudot|arXiv (Cornell University)|Nov 22, 2010

Advanced Image and Video Retrieval Techniques31 references3 citations

TL;DR

This paper presents a provably correct and efficient algorithm for reverse nearest neighbors (RNN) search in high-dimensional spaces using Locality-Sensitive Hashing (LSH). It reduces RNN queries to one ε-NN search and a controlled number of exhaustive r-PLEB queries, achieving sublinear query time with output-sensitive complexity dependent on the size of the RNN set and a condition number reflecting instance difficulty.

ABSTRACT

We investigate the problem of finding reverse nearest neighbors efficiently. Although provably good solutions exist for this problem in low or fixed dimensions, to this date the methods proposed in high dimensions are mostly heuristic. We introduce a method that is both provably correct and efficient in all dimensions, based on a reduction of the problem to one instance of $\e$-nearest neighbor search plus a controlled number of instances of {\em exhaustive $r$-\pleb}, a variant of {\em Point Location among Equal Balls} where all the $r$-balls centered at the data points that contain the query point are sought for, not just one. The former problem has been extensively studied and elegantly solved in high dimensions using Locality-Sensitive Hashing (LSH) techniques. By contrast, the latter problem has a complexity that is still not fully understood. We revisit the analysis of the LSH scheme for exhaustive $r$-\pleb using a somewhat refined notion of locality-sensitive family of hash function, which brings out a meaningful output-sensitive term in the complexity of the problem. Our analysis, combined with a non-isometric lifting of the data, enables us to answer exhaustive $r$-\pleb queries (and down the road reverse nearest neighbors queries) efficiently. Along the way, we obtain a simple algorithm for answering exact nearest neighbor queries, whose complexity is parametrized by some {\em condition number} measuring the inherent difficulty of a given instance of the problem.

Motivation & Objective

• To address the lack of theoretically grounded, efficient solutions for reverse nearest neighbors (RNN) search in high-dimensional spaces, where existing methods are mostly heuristic.
• To overcome the curse of dimensionality in RNN queries, which inherently return sets of points whose size can be exponential in dimension.
• To achieve a query time of the form Õ(n^ϱ + |RNN_P(q)|) with ϱ < 1, enabling sublinear performance while being sensitive to the actual output size.
• To refine the analysis of LSH for exhaustive r-PLEB by introducing a notion of locality-sensitive families with output-sensitive complexity.
• To develop a non-isometric lifting technique that reduces the output-sensitive exponent α in query time, improving efficiency for hard instances.

Proposed method

• Reduces the bichromatic RNN query to one instance of ε-NN search and a poly-logarithmic number of exhaustive r-PLEB queries.
• Applies a non-isometric lifting of the data to improve the efficiency of the exhaustive r-PLEB phase, reducing the output-sensitive exponent α.
• Uses a refined analysis of LSH families for exhaustive r-PLEB, introducing a condition number that captures instance-specific difficulty.
• Employs s-stable distributions (for ℓ_s norms) to define the LSH scheme, with parameters p₀, p₁, p₂ derived from the cumulative distribution function Φ.
• Introduces a new algorithm (Algorithm 6) that combines LSH-based ε-NN and r-PLEB queries to answer RNN queries with high probability.
• Derives query time bounds that are output-sensitive and parametrized by ϱ and α, where ϱ < 1 and α ≤ εϱ < ε for ℓ₁ and ℓ₂ norms.

Experimental results

Research questions

• RQ1 Can RNN search in high dimensions be solved with provable correctness and sublinear query time, avoiding purely heuristic approaches?
• RQ2 How can the complexity of exhaustive r-PLEB queries be analyzed more precisely, especially in terms of output sensitivity?
• RQ3 Can the output-sensitive exponent α in the query time be reduced below ε, and what trade-offs arise from doing so?
• RQ4 To what extent can non-isometric data lifting improve the efficiency of RNN and r-PLEB queries in high-dimensional spaces?
• RQ5 Is it possible to achieve near-optimal complexity bounds for RNN search, and how close are the current bounds to the theoretical optimum?

Key findings

• The proposed method answers bichromatic RNN queries in expected Õ(1/ε · n^ϱ + n^α · |ε(2+ε)-RNN_B,Y(q)|) time, with ϱ < 1 and α ≤ εϱ < ε for ℓ₁ and ℓ₂ norms.
• For ℓ₁, ϱ ≤ 1/(1 + min{ε², √ε}/4) < 1, and for ℓ₂, ϱ ≤ 1/(1 + ε²/(1+ε)) < 1, ensuring sublinear query time.
• The space complexity is Õ(1/ε · n^{1+ϱ} + n²), which is polynomial and acceptable for practical use.
• The method introduces a new exact nearest neighbor query algorithm whose complexity depends on a condition number reflecting the difficulty of the instance.
• Non-isometric lifting allows reducing α to arbitrarily small positive constants, though at the cost of increasing ϱ toward 1.
• The analysis reveals that the output-sensitive term n^α is bounded by n^ε, suggesting that the method scales well with the size of the RNN set.