[Paper Review] High-Performance Reachability Query Processing under Index Size Restrictions

Stephan Seufert, Avishek Anand+2 more|arXiv (Cornell University)|Nov 14, 2012

Data Management and Algorithms9 references3 citations

TL;DR

This paper proposes FERRARI, a space-adaptive index structure for high-performance reachability query processing in large-scale directed graphs under strict index size constraints. By adaptively compressing the transitive closure through selective interval merging and guided online search, FERRARI achieves sub-microsecond query times on real-world web-scale graphs while outperforming prior methods like GRAIL, especially on positive reachability queries.

ABSTRACT

In this paper, we propose a scalable and highly efficient index structure for the reachability problem over graphs. We build on the well-known node interval labeling scheme where the set of vertices reachable from a particular node is compactly encoded as a collection of node identifier ranges. We impose an explicit bound on the size of the index and flexibly assign approximate reachability ranges to nodes of the graph such that the number of index probes to answer a query is minimized. The resulting tunable index structure generates a better range labeling if the space budget is increased, thus providing a direct control over the trade off between index size and the query processing performance. By using a fast recursive querying method in conjunction with our index structure, we show that in practice, reachability queries can be answered in the order of microseconds on an off-the-shelf computer - even for the case of massive-scale real world graphs. Our claims are supported by an extensive set of experimental results using a multitude of benchmark and real-world web-scale graph datasets.

Motivation & Objective

• To address the challenge of efficient reachability query processing in massive-scale graphs where index size is constrained and main memory is limited.
• To overcome the performance bottleneck of existing size-constrained indices, particularly on positive reachability queries, which require costly recursive exploration.
• To provide a tunable trade-off between index size and query performance by flexibly assigning approximate reachability intervals under a user-defined space budget.
• To design an index structure that minimizes expected query processing time while maintaining high accuracy and scalability on real-world datasets.

Proposed method

• The method employs a node interval labeling scheme, encoding reachable vertices as identifier ranges to compactly represent reachability sets.
• It adaptively merges adjacent intervals during index construction when the space budget is exceeded, balancing compression and query efficiency.
• The approach models the interval assignment as an interval cover problem, solved efficiently to minimize query cost under size constraints.
• A fast recursive querying mechanism is used in conjunction with the index to enable guided search and accelerate both positive and negative reachability queries.
• The index supports both exact and approximate reachability ranges, with the ability to scale performance by increasing the space budget.
• The construction phase uses heuristics to prioritize interval merging strategies that reduce expected number of index probes per query.

Experimental results

Research questions

• RQ1 Can a space-constrained reachability index be constructed to minimize expected query processing time while maintaining high accuracy?
• RQ2 How does adaptive interval compression affect query performance on positive reachability queries compared to existing methods?
• RQ3 To what extent can the index structure scale in terms of query speed and storage efficiency across diverse real-world graph workloads?
• RQ4 Does the guided online search procedure significantly reduce the number of index probes required for reachability evaluation?

Key findings

• On the YAGO2 dataset, FERRARI-L achieved a median query time of 10.45ms for 100k random queries, outperforming GRAIL (30.53ms) and reducing query time by 65.7%.
• For positive queries on the Twitter dataset, FERRARI-L reduced median query time to 9.80ms, compared to 18.21ms for GRAIL, achieving a 46.1% improvement.
• On the GovWild dataset, FERRARI-L achieved a median query time of 13.33ms for positive queries, significantly lower than GRAIL’s 29.84ms, a 55.3% reduction.
• FERRARI maintained a compact index size, with YAGO2’s index at 5,844.87KB—well below GRAIL’s 59,587KB—while delivering superior performance.
• Construction time for FERRARI was consistently lower than GRAIL across all datasets, with FERRARI-L taking only 137.88ms on YAGO2 versus GRAIL’s 182.96ms.
• On the Web-UK dataset, FERRARI-L achieved a median query time of 16.85ms for 100k positive queries, outperforming GRAIL (26,792.72ms) by a factor of over 1,500x.