[Paper Review] A fast nearest neighbor search algorithm based on vector quantization

Sylvain Corlay|arXiv (Cornell University)|May 25, 2011

Data Management and Algorithms14 references3 citations

TL;DR

This paper proposes a novel fast nearest neighbor search algorithm using vector quantization to recursively partition data into clusters, forming a 'quantization tree' that leverages Voronoi diagrams and Delaunay triangulations. It introduces the 'friend Voronoi cells' technique to minimize unnecessary node exploration, significantly improving performance in high dimensions (d > 5), where it outperforms Kd-tree and principal axis tree on large-scale datasets with up to 52 seconds for 10M queries in d=8.

ABSTRACT

In this article, we propose a new fast nearest neighbor search algorithm, based on vector quantization. Like many other branch and bound search algorithms [1,10], a preprocessing recursively partitions the data set into disjointed subsets until the number of points in each part is small enough. In doing so, a search-tree data structure is built. This preliminary recursive data-set partition is based on the vector quantization of the empirical distribution of the initial data-set. Unlike previously cited methods, this kind of partitions does not a priori allow to eliminate several brother nodes in the search tree with a single test. To overcome this difficulty, we propose an algorithm to reduce the number of tested brother nodes to a minimal list that we call "friend Voronoi cells". The complete description of the method requires a deeper insight into the properties of Delaunay triangulations and Voronoi diagrams

Motivation & Objective

• To address the curse of dimensionality in nearest neighbor search by leveraging data geometry through vector quantization.
• To overcome the inefficiency of tree-based search algorithms in high dimensions by reducing redundant node exploration.
• To develop a preprocessing method that accelerates nearest neighbor queries via geometric data structure optimization.
• To introduce a novel 'friend Voronoi cell' algorithm that minimizes the number of candidate nodes to explore during search.
• To evaluate the performance of the quantization tree with and without the friend node optimization on real and synthetic datasets.

Proposed method

• The algorithm recursively partitions the data using vector quantization based on the empirical distribution, forming a hierarchical tree structure.
• It constructs a Voronoi diagram from the centroids of quantized clusters to define cell boundaries for nearest neighbor assignment.
• The method uses Delaunay triangulation to identify geometric relationships between sites and compute 'friend Voronoi cells' that may contain the nearest neighbor.
• A 'friend node' algorithm is introduced to reduce the number of sibling nodes that must be explored by identifying only the most relevant candidates via circumsphere and visibility checks.
• The preprocessing phase includes Delaunay triangulation and pseudo-insertion to compute visibility and neighbor relationships for efficient query routing.
• The algorithm is implemented in C++ with optimized computational geometry routines for Voronoi and Delaunay structures.

Experimental results

Research questions

• RQ1 Can vector quantization-based recursive partitioning improve nearest neighbor search efficiency compared to traditional tree-based methods?
• RQ2 How can the number of explored sibling nodes in a search tree be minimized without sacrificing correctness?
• RQ3 What is the impact of the 'friend Voronoi cell' technique on query time in high-dimensional spaces?
• RQ4 In what dimensionality range does the quantization tree outperform Kd-tree and principal axis tree?
• RQ5 How does the computational cost of preprocessing the friend cell structure scale with dimensionality and dataset size?

Key findings

• In dimension d=8, the quantization tree achieved a query time of 52.31 seconds for 10 million queries on a 5,000-point Gaussian dataset, outperforming the principal axis tree (82.30s) and Kd-tree (118.93s).
• For uniform data in d=8, the quantization tree required 32.62 seconds, compared to 47.53 seconds for the principal axis tree and 55.71 seconds for the Kd-tree.
• The quantization tree showed superior performance in dimensions d > 5, where it significantly reduced query time relative to baseline methods.
• The friend node algorithm provided no competitive advantage in dimensions higher than 7 due to increasing preprocessing cost and larger friend cell lists.
• In low dimensions (d=2,3), the quantization tree performed comparably to the principal axis tree and Kd-tree, with moderate gains in d=4.
• The preprocessing cost of computing Delaunay triangulations and friend cells becomes prohibitive in high dimensions, limiting the use of the friend node optimization beyond d=7.