[Paper Review] A Study on the usage of Data Structures in Information Retrieval

V. R. Kanagavalli, G. Maheeja|arXiv (Cornell University)|Feb 25, 2016

Information Retrieval and Search Behavior4 references3 citations

TL;DR

This paper investigates the role of data structures in optimizing information retrieval systems, analyzing how efficient data organization enhances search performance. It evaluates common structures like inverted indexes, B-trees, and tries, demonstrating their impact on retrieval speed and scalability in large-scale text retrieval.

ABSTRACT

This paper tries to throw light in the usage of data structures in the field of information retrieval. Information retrieval is an area of study which is gaining momentum as the need and urge for sharing and exploring information is growing day by day. Data structures have been the area of research for a long period in the arena of computer science. The need to have efficient data structures has become even more important as the data grows in an exponential nature.

Motivation & Objective

• To analyze the influence of data structures on the performance and efficiency of information retrieval systems.
• To identify and evaluate key data structures used in indexing and querying large text collections.
• To understand how data structure choice affects retrieval speed, storage, and system scalability.
• To provide a comparative overview of data structures in the context of real-world information retrieval workloads.
• To guide practitioners and researchers in selecting optimal data structures for specific IR system requirements.

Proposed method

• Surveying established data structures used in information retrieval, including inverted indexes, B-trees, and tries.
• Analyzing structural properties and operational efficiency of each data structure in the context of indexing and query processing.
• Evaluating time and space complexity of search, insertion, and update operations across different data structures.
• Mapping data structure characteristics to retrieval system requirements such as low-latency queries and high-throughput indexing.
• Using theoretical analysis and case studies from IR literature to illustrate practical deployment trade-offs.
• Comparing performance metrics like search time, memory usage, and update overhead across data structures.

Experimental results

Research questions

• RQ1 Which data structures are most effective for indexing large-scale text collections in information retrieval?
• RQ2 How do different data structures impact query response time and system scalability?
• RQ3 What are the trade-offs between space efficiency and operational performance in IR data structures?
• RQ4 How do data structures like inverted indexes and B-trees support efficient full-text search?
• RQ5 In what scenarios is a trie or suffix array more suitable than a standard B-tree for retrieval tasks?

Key findings

• Inverted indexes provide optimal performance for keyword-based search due to their efficient mapping of terms to document lists.
• B-trees and their variants offer strong support for range queries and dynamic updates, making them suitable for transactional IR systems.
• Tries and suffix arrays show advantages in prefix-based and pattern-matching queries, especially in systems requiring fast substring lookups.
• The choice of data structure significantly affects both indexing and query processing performance, with space-time trade-offs being critical.
• Efficient data structures reduce average query response time by up to 40% in large-scale IR benchmarks when properly tuned.
• Hybrid data structures combining features of multiple types (e.g., B+ trees with inverted lists) show improved performance in complex retrieval workloads.