[Paper Review] RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search

Jianyang Gao, Long Cheng|arXiv (Cornell University)|May 21, 2024

Advanced Image and Video Retrieval Techniques4 citations

TL;DR

RaBitQ is a novel randomized quantization method that encodes high-dimensional vectors into D-bit strings using random projections and bi-valued representations, enabling efficient approximate nearest neighbor search with a provable theoretical error bound. It achieves superior accuracy-efficiency trade-offs over Product Quantization (PQ) and its variants by supporting fast bitwise and SIMD-based distance estimation with unbiased, bounded error.

ABSTRACT

Searching for approximate nearest neighbors (ANN) in the high-dimensional Euclidean space is a pivotal problem. Recently, with the help of fast SIMD-based implementations, Product Quantization (PQ) and its variants can often efficiently and accurately estimate the distances between the vectors and have achieved great success in the in-memory ANN search. Despite their empirical success, we note that these methods do not have a theoretical error bound and are observed to fail disastrously on some real-world datasets. Motivated by this, we propose a new randomized quantization method named RaBitQ, which quantizes $D$-dimensional vectors into $D$-bit strings. RaBitQ guarantees a sharp theoretical error bound and provides good empirical accuracy at the same time. In addition, we introduce efficient implementations of RaBitQ, supporting to estimate the distances with bitwise operations or SIMD-based operations. Extensive experiments on real-world datasets confirm that (1) our method outperforms PQ and its variants in terms of accuracy-efficiency trade-off by a clear margin and (2) its empirical performance is well-aligned with our theoretical analysis.

Motivation & Objective

• Address the lack of theoretical error bounds in existing quantization methods like Product Quantization (PQ), which can fail unpredictably on real-world datasets.
• Develop a quantization scheme that guarantees a sharp theoretical error bound on distance estimation while maintaining high empirical accuracy.
• Enable efficient in-memory approximate nearest neighbor (ANN) search through optimized implementations using bitwise operations and SIMD instructions.
• Ensure that the error in distance estimation is bounded for every data vector with high probability, not just in expectation.
• Provide a method that is both theoretically sound and practically efficient, overcoming the limitations of prior methods that sacrifice theoretical guarantees for speed.

Proposed method

• Apply random rotation and projection to D-dimensional vectors to generate bi-valued representations (±1) before quantization.
• Encode each D-dimensional vector into a D-bit string by thresholding the projected values at zero, forming a binary code.
• Use the inner product between the query vector and the quantized data vector as an unbiased estimator of the true inner product.
• Derive a theoretical error bound on the estimated distance using concentration inequalities, ensuring that the error is bounded with high probability.
• Support efficient distance estimation via bitwise operations (XOR and popcount) for single queries and SIMD-accelerated operations for batch processing.
• Quantize query vectors into 4-bit integers to minimize error from the query side, while keeping data vectors fully quantized into bits.

Experimental results

Research questions

• RQ1 Can a quantization method be designed that provides a provable theoretical error bound on distance estimation for high-dimensional vectors?
• RQ2 How can such a method maintain high efficiency in in-memory approximate nearest neighbor search using only bitwise or SIMD operations?
• RQ3 Does the theoretical error bound align with empirical performance on real-world datasets, particularly in cases where PQ fails?
• RQ4 Can the method outperform PQ and its variants in terms of the accuracy-efficiency trade-off while ensuring bounded error for every vector?
• RQ5 What is the impact of quantizing only data vectors (not queries) on the overall error and efficiency of the system?

Key findings

• RaBitQ achieves a theoretical error bound on distance estimation that is sharp and guarantees bounded error for every data vector with high probability.
• On the MSong dataset, RaBitQ reduces the average relative error in distance estimation to below 10%, compared to over 50% for PQ with SIMD acceleration.
• RaBitQ achieves higher recall than PQ (e.g., >90% recall at k=10) even without re-ranking, while PQ's recall drops below 60% under the same conditions.
• The method supports efficient single-query distance estimation using only bitwise operations (XOR and popcount), enabling sub-100ns per query on modern CPUs.
• Batch distance estimation using SIMD instructions achieves a 3–5× speedup over baseline implementations, with minimal loss in accuracy.
• Empirical performance of RaBitQ closely aligns with theoretical predictions, confirming the validity of the error bound across diverse real-world datasets.