[Paper Review] A Survey of Learned Indexes for the Multi-dimensional Space

Abdullah Al Mamun, Hao Wu+3 more|arXiv (Cornell University)|Mar 11, 2024

Image Retrieval and Classification Techniques4 citations

TL;DR

This survey presents a comprehensive taxonomy and analysis of learned multi-dimensional indexes, categorizing 43 such indexes based on learning approach, data layout, and query support. It identifies key challenges in extending learned indexing to multi-dimensional data and outlines open research directions in security, benchmarking, and theoretical analysis.

ABSTRACT

A recent research trend involves treating database index structures as Machine Learning (ML) models. In this domain, single or multiple ML models are trained to learn the mapping from keys to positions inside a data set. This class of indexes is known as "Learned Indexes." Learned indexes have demonstrated improved search performance and reduced space requirements for one-dimensional data. The concept of one-dimensional learned indexes has naturally been extended to multi-dimensional (e.g., spatial) data, leading to the development of "Learned Multi-dimensional Indexes". This survey focuses on learned multi-dimensional index structures. Specifically, it reviews the current state of this research area, explains the core concepts behind each proposed method, and classifies these methods based on several well-defined criteria. We present a taxonomy that classifies and categorizes each learned multi-dimensional index, and survey the existing literature on learned multi-dimensional indexes according to this taxonomy. Additionally, we present a timeline to illustrate the evolution of research on learned indexes. Finally, we highlight several open challenges and future research directions in this emerging and highly active field.

Motivation & Objective

• To provide a systematic classification of learned multi-dimensional index structures using a novel taxonomy.
• To analyze the core techniques and design choices in existing learned multi-dimensional indexes.
• To identify open challenges in security, benchmarking, and theoretical foundations for learned multi-dimensional indexing.
• To highlight the evolution and current state of research in learned multi-dimensional indexing compared to one-dimensional counterparts.
• To guide future research by outlining key opportunities in GPU acceleration, robustness, and real-world workload evaluation.

Proposed method

• Proposes a seven-dimensional taxonomy to classify learned multi-dimensional indexes based on learning vs. indexing models, data layout immutability, and insertion strategies.
• Categorizes methods by whether they learn the index directly or index pre-trained ML models, and by their use of in-place or delta-buffer insertion.
• Reviews 43 learned multi-dimensional index structures, analyzing their supported query types and underlying machine learning models.
• Introduces a timeline to illustrate the evolution of learned indexing research from one-dimensional to multi-dimensional extensions.
• Examines hybrid approaches that combine traditional indexes with ML models for improved performance and robustness.
• Evaluates the impact of dimensionality on model choice and error-correction mechanisms, particularly in the absence of total ordering in multi-dimensional spaces.

Experimental results

Research questions

• RQ1 How can learned multi-dimensional indexes be systematically classified based on design and operational characteristics?
• RQ2 What are the key challenges in adapting one-dimensional learned indexing techniques to multi-dimensional data?
• RQ3 How do different learning strategies (e.g., pure learned vs. hybrid) affect performance and space efficiency in multi-dimensional indexing?
• RQ4 What are the open research gaps in security, benchmarking, and theoretical analysis for learned multi-dimensional indexes?
• RQ5 To what extent can GPU acceleration and real-world workloads improve the evaluation and deployment of learned multi-dimensional indexes?

Key findings

• Learned multi-dimensional indexes face unique challenges due to the lack of total ordering in multi-dimensional spaces, complicating error-correction and layout design.
• The taxonomy proposed in the paper enables systematic comparison and classification of 43 learned multi-dimensional index structures based on seven key criteria.
• While pure learned indexes show promise in one-dimensional settings, extending them to multi-dimensional data requires new strategies for data layout and model generalization.
• Security vulnerabilities, such as model poisoning attacks, can severely degrade performance, highlighting the need for robust learning mechanisms in production systems.
• Comprehensive benchmarks using real data and workloads are currently missing, creating a significant gap in evaluating the true performance of learned multi-dimensional indexes.
• Theoretical analysis of learned one-dimensional indexes has shown provable performance gains (e.g., O(log log n) query time), but such analysis remains limited for multi-dimensional variants.