[Paper Review] OOD-DiskANN: Efficient and Scalable Graph ANNS for Out-of-Distribution Queries

Shikhar Jaiswal, Ravishankar Krishnaswamy+3 more|arXiv (Cornell University)|Oct 22, 2022

Advanced Image and Video Retrieval Techniques6 citations

TL;DR

OOD-DiskANN proposes a modified DiskANN graph-based Approximate Nearest Neighbor Search (ANNS) index that efficiently handles out-of-distribution (OOD) queries by leveraging a small sample of OOD queries during index construction. By integrating query-aware product quantization and a parallelized graph reordering algorithm, it achieves up to 40% lower latency than state-of-the-art methods for OOD queries while maintaining high recall and comparable build-time and memory usage.

ABSTRACT

State-of-the-art algorithms for Approximate Nearest Neighbor Search (ANNS) such as DiskANN, FAISS-IVF, and HNSW build data dependent indices that offer substantially better accuracy and search efficiency over data-agnostic indices by overfitting to the index data distribution. When the query data is drawn from a different distribution - e.g., when index represents image embeddings and query represents textual embeddings - such algorithms lose much of this performance advantage. On a variety of datasets, for a fixed recall target, latency is worse by an order of magnitude or more for Out-Of-Distribution (OOD) queries as compared to In-Distribution (ID) queries. The question we address in this work is whether ANNS algorithms can be made efficient for OOD queries if the index construction is given access to a small sample set of these queries. We answer positively by presenting OOD-DiskANN, which uses a sparing sample (1% of index set size) of OOD queries, and provides up to 40% improvement in mean query latency over SoTA algorithms of a similar memory footprint. OOD-DiskANN is scalable and has the efficiency of graph-based ANNS indices. Some of our contributions can improve query efficiency for ID queries as well.

Motivation & Objective

• To address the significant performance degradation in ANNS systems when queries are out-of-distribution (OOD), such as cross-modal text-to-image retrieval.
• To improve search efficiency for OOD queries without sacrificing in-distribution (ID) query performance or increasing memory and build-time overhead.
• To develop a scalable, disk-based ANNS system that adapts to OOD query distributions using only a small sample of such queries during index construction.
• To enhance product quantization schemes for OOD scenarios by introducing query-aware and adaptive pivot learning.
• To parallelize and optimize the graph reordering step in DiskANN for disk-resident, large-scale ANNS workloads.

Proposed method

• Extends DiskANN's graph construction to incorporate a small sample (1% of index size) of OOD queries during index building, enabling better routing for OOD queries.
• Introduces query-aware product quantization (APQ) and adaptive OQ (AOPQ) to learn quantization centroids that minimize reconstruction error for both base data and OOD queries.
• Adapts and parallelizes the state-of-the-art graph reordering algorithm to reduce I/O and improve search efficiency in disk-based ANNS systems.
• Uses Mahalanobis distance histograms as a proxy to identify OOD query sets, enabling systematic evaluation of distributional shift.
• Employs a beam search with a beam width of 4 during search, and uses 16 search threads to optimize latency on multi-core hardware.
• Maintains a fixed RAM budget of 200 GB and uses 8-bit product quantization with M = D or M = D/4 depending on in-memory or disk-resident storage.

Experimental results

Research questions

• RQ1 Can a small sample of OOD queries be used to significantly improve ANNS efficiency for out-of-distribution queries without degrading in-distribution performance?
• RQ2 How can product quantization be adapted to reduce quantization error for queries drawn from a different distribution than the base data?
• RQ3 What is the impact of parallelized graph reordering on I/O and latency in disk-based ANNS systems for OOD queries?
• RQ4 To what extent can the performance gap between ID and OOD query latencies be closed using index construction techniques sensitive to query distribution?
• RQ5 Can the proposed method achieve state-of-the-art recall and latency trade-offs for OOD queries while maintaining memory and build-time efficiency?

Key findings

• OOD-DiskANN reduces mean query latency by 10–25% for OOD queries compared to state-of-the-art ANNS methods, with up to 40% improvement in latency for the same recall target.
• The method achieves a 15–40% reduction in I/O requests during search, directly contributing to lower latency.
• The improvement is achieved with only a 7–12% increase in build-time and no increase in peak RAM or disk usage.
• Query-aware product quantization (APQ) and adaptive OQ (AOPQ) significantly reduce quantization error for OOD queries, improving search efficiency.
• The parallelized graph reordering algorithm reduces I/O and maintains high performance on disk-resident, large-scale datasets.
• On three large-scale datasets—Yandex Text-to-Image-1B, Turing Text-to-Image, and Web Ads—the method consistently outperforms existing ANNS systems for OOD queries.