[Paper Review] Optimal Las Vegas Locality Sensitive Data Structures

Thomas D. Ahle|arXiv (Cornell University)|Apr 6, 2017

Advanced Image and Video Retrieval Techniques4 citations

TL;DR

This paper presents the first Las Vegas locality-sensitive data structures for approximate near neighbor search in high-dimensional spaces, achieving query and space complexity matching state-of-the-art data-independent Locality Sensitive Hashing (LSH), while guaranteeing no false negatives. The approach combines brute force, tensoring, perfect hashing, and splitter functions to construct deterministic space partitions, enabling optimal performance for Hamming, ℓ₁, ℓ₂, and Jaccard similarity search with sub-linear query time and correctness guarantees.

ABSTRACT

We show that approximate similarity (near neighbour) search can be solved in high dimensions with performance matching state of the art (data independent) Locality Sensitive Hashing, but with a guarantee of no false negatives. Specifically, we give two data structures for common problems. For $c$-approximate near neighbour in Hamming space we get query time $dn^{1/c+o(1)}$ and space $dn^{1+1/c+o(1)}$ matching that of \cite{indyk1998approximate} and answering a long standing open question from~\cite{indyk2000dimensionality} and~\cite{pagh2016locality} in the affirmative. By means of a new deterministic reduction from $\ell_1$ to Hamming we also solve $\ell_1$ and $\ell_2$ with query time $d^2n^{1/c+o(1)}$ and space $d^2 n^{1+1/c+o(1)}$. For $(s_1,s_2)$-approximate Jaccard similarity we get query time $dn^{ρ+o(1)}$ and space $dn^{1+ρ+o(1)}$, $ρ=\log\frac{1+s_1}{2s_1}\big/\log\frac{1+s_2}{2s_2}$, when sets have equal size, matching the performance of~\cite{tobias2016}. The algorithms are based on space partitions, as with classic LSH, but we construct these using a combination of brute force, tensoring, perfect hashing and splitter functions à la~\cite{naor1995splitters}. We also show a new dimensionality reduction lemma with 1-sided error.

Motivation & Objective

• To resolve a long-standing open problem in high-dimensional similarity search: whether Las Vegas data structures could match the performance of Monte Carlo LSH while guaranteeing no false negatives.
• To construct deterministic, Las Vegas versions of approximate near neighbor data structures for Hamming space, ℓ₁, ℓ₂, and Jaccard similarity, with query and space complexity matching the best-known LSH methods.
• To develop a new dimensionality reduction technique with 1-sided error, enabling efficient transformation from ℓ₁ and ℓ₂ to Hamming space without false negatives.
• To provide explicit, constructive methods for building large Turán systems and covering codes as byproducts of the main construction.

Proposed method

• Construct space partitions using a combination of brute force, tensoring, perfect hashing, and splitter functions inspired by Naor et al. (1995), ensuring deterministic and correct partitioning.
• Use a novel deterministic reduction from ℓ₁ to Hamming space, enabling the extension of Hamming-based Las Vegas data structures to ℓ₁ and ℓ₂ metrics.
• Apply a new dimensionality reduction lemma with 1-sided error to map ℓ₁ and ℓ₂ vectors to Hamming space while preserving similarity guarantees.
• Leverage the Locality Sensitive Filter (LSF) framework to design data structures with sub-linear query time and space usage matching LSH bounds.
• Introduce a modified analysis of binomial coefficients and Vandermonde convolution to bound the number of false positives in the construction, ensuring correctness and efficiency.
• Use integrality adjustments (e.g., rounding r to nearest square, adjusting k to ensure divisibility) to make combinatorial parameters integral without significantly affecting asymptotic performance.

Experimental results

Research questions

• RQ1 Can Las Vegas data structures be constructed for approximate near neighbor search in high-dimensional spaces with query and space complexity matching that of state-of-the-art LSH?
• RQ2 Is it possible to achieve optimal performance for ℓ₁ and ℓ₂ similarity search using Las Vegas guarantees, without relying on Monte Carlo randomness?
• RQ3 Can a deterministic reduction from ℓ₁ to Hamming space be constructed that preserves similarity and enables efficient Las Vegas search?
• RQ4 Can the overhead term in the exponent (e.g., (log n)^{-1/4}) be reduced or eliminated in Las Vegas LSH constructions?
• RQ5 Are there general techniques to derandomize existing LSH algorithms while preserving optimal asymptotic performance and correctness?

Key findings

• For c-approximate near neighbor in Hamming space, the paper achieves query time dn^{1/c + o(1)} and space dn^{1 + 1/c + o(1)}, matching the performance of Indyk and Motwani (1998) and resolving an open question from Indyk (2000a) and Pagh (2016).
• For ℓ₁ and ℓ₂ similarity, the method achieves query time d²n^{1/c + o(1)} and space d²n^{1 + 1/c + o(1)}, matching the best-known LSH bounds and enabling 1-sided error dimensionality reduction.
• For (s₁,s₂)-approximate Jaccard similarity with equal-sized sets, the data structure achieves query time dn^{ρ + o(1)} and space dn^{1 + ρ + o(1)}, where ρ = log((1+s₁)/(2s₁)) / log((1+s₂)/(2s₂)), matching the performance of Pagh and Christiani (2017).
• The construction introduces a new deterministic reduction from ℓ₁ to Hamming space, enabling the use of Hamming-based Las Vegas structures for ℓ₁ and ℓ₂ metrics.
• The paper provides explicit, constructive methods for large Turán systems and covering codes for pairs, as byproducts of the main construction.
• The approach introduces a novel dimensionality reduction lemma with 1-sided error, enabling efficient and correct transformation from ℓ₁ to Hamming space without false negatives.