[Paper Review] Optimal lower bounds for locality sensitive hashing (except when q is tiny)

Ryan O’Donnell, Yi Wu+1 more|ArXiv.org|Dec 1, 2009

Advanced Image and Video Retrieval Techniques3 citations

TL;DR

This paper establishes an optimal lower bound of $\rho \geq 1/c - o_d(1)$ for Locality Sensitive Hashing (LSH) in the Hamming space $\{0,1\}^d$ under the standard assumption that $q$ is bounded away from zero (i.e., not $2^{-\Theta(d)}$). The proof relies on the log-convexity of noise stability of Boolean functions, showing that the previously known lower bound of $\sim 0.46/c$ is suboptimal and that the upper bound $\rho \leq 1/c$ from Indyk and Motwani is tight up to lower-order terms.

ABSTRACT

We study lower bounds for Locality Sensitive Hashing (LSH) in the strongest setting: point sets in {0,1}^d under the Hamming distance. Recall that here H is said to be an (r, cr, p, q)-sensitive hash family if all pairs x, y in {0,1}^d with dist(x,y) at most r have probability at least p of collision under a randomly chosen h in H, whereas all pairs x, y in {0,1}^d with dist(x,y) at least cr have probability at most q of collision. Typically, one considers d tending to infinity, with c fixed and q bounded away from 0. For its applications to approximate nearest neighbor search in high dimensions, the quality of an LSH family H is governed by how small its "rho parameter" rho = ln(1/p)/ln(1/q) is as a function of the parameter c. The seminal paper of Indyk and Motwani showed that for each c, the extremely simple family H = {x -> x_i : i in d} achieves rho at most 1/c. The only known lower bound, due to Motwani, Naor, and Panigrahy, is that rho must be at least .46/c (minus o_d(1)). In this paper we show an optimal lower bound: rho must be at least 1/c (minus o_d(1)). This lower bound for Hamming space yields a lower bound of 1/c^2 for Euclidean space (or the unit sphere) and 1/c for the Jaccard distance on sets; both of these match known upper bounds. Our proof is simple; the essence is that the noise stability of a boolean function at e^{-t} is a log-convex function of t.

Motivation & Objective

• To close the gap between the known upper bound $\rho \leq 1/c$ and the prior lower bound of $\sim 0.46/c$ for LSH in Hamming space.
• To establish that the $\rho \leq 1/c$ bound from Indyk and Motwani is asymptotically optimal for $q$ bounded away from zero.
• To provide a clean, conceptual proof based on the log-convexity of noise stability of Boolean functions.
• To clarify the range of $q$ for which $\rho$ accurately reflects LSH quality, excluding trivial cases where $q$ is exponentially small.

Proposed method

• The proof uses the noise stability of a Boolean function at $e^{-t}$, showing it is a log-convex function of $t$, which is central to the lower bound argument.
• It constructs a distribution over hash functions and analyzes collision probabilities for pairs at distance $r$ and $cr$, using concentration bounds.
• Chernoff bounds are applied to estimate tail probabilities $e_1$ and $e_2$, which quantify deviations in binomial random variables related to Hamming weights.
• The analysis introduces a parameter $\Delta$ to control error terms and optimizes over $\Delta$ to minimize the total error $e$ in the $\rho$ lower bound.
• The bound is shown to hold under the condition that $q$ is not tiny (i.e., not $2^{-\Theta(d)}$), which is necessary to avoid trivial $\rho = 0$ constructions.
• The result is extended to Euclidean space and Jaccard distance, yielding $\rho \geq 1/c^2$ and $\rho \geq 1/c$ respectively, matching known upper bounds.

Experimental results

Research questions

• RQ1 Is the $\rho \leq 1/c$ upper bound for LSH in Hamming space tight, or can a smaller $\rho$ be achieved?
• RQ2 Can the prior lower bound of $\sim 0.46/c$ be improved, and if so, by how much?
• RQ3 Does the $\rho$ parameter accurately reflect LSH quality when $q$ is very small (e.g., $2^{-\Theta(d)}$)?
• RQ4 Can a clean, structural proof based on noise stability properties establish optimality of the $1/c$ bound?
• RQ5 What is the correct asymptotic lower bound for $\rho$ in terms of $c$ for $q$ bounded away from zero?

Key findings

• The paper establishes a tight lower bound of $\rho \geq 1/c - o_d(1)$ for LSH in $\{0,1\}^d$ under Hamming distance, matching the upper bound from Indyk and Motwani.
• The lower bound holds under the condition that $q$ is not exponentially small, i.e., $q = 2^{-o(d)}$, which is the regime where $\rho$ is meaningful for LSH quality.
• The proof technique relies on the log-convexity of noise stability of Boolean functions, a novel and conceptual approach to LSH lower bounds.
• For Euclidean space, the result implies $\rho \geq 1/c^2$, matching the known upper bound and closing the gap.
• For the Jaccard distance on sets, the bound yields $\rho \geq 1/c$, which matches the known upper bound.
• The paper shows that the prior lower bound of $\sim 0.46/c$ is suboptimal and that $1/c$ is the correct asymptotic limit.