[Paper Review] Search and Navigation in Relational Databases

Richard Wheeldon, Mark Levene+1 more|ArXiv.org|Jul 31, 2003

Advanced Database Systems and Queries29 references6 citations

TL;DR

This paper presents DbSurfer, a system that enables keyword search and navigational discovery in relational databases by automatically constructing Memex-like trails through foreign key relationships. It uses a novel join discovery algorithm to traverse the schema graph and retrieve relevant data via primary-key-only queries, achieving fast response times without requiring SQL expertise.

ABSTRACT

We present a new application for keyword search within relational databases, which uses a novel algorithm to solve the join discovery problem by finding Memex-like trails through the graph of foreign key dependencies. It differs from previous efforts in the algorithms used, in the presentation mechanism and in the use of primary-key only database queries at query-time to maintain a fast response for users. We present examples using the DBLP data set.

Motivation & Objective

• To address the challenge of making relational database content accessible to end-users who lack SQL expertise.
• To overcome the limitations of traditional search engines by indexing the 'deep web' content stored in relational databases.
• To provide a user-friendly interface that combines keyword search with navigational trails, similar to hypertext systems.
• To enable efficient, fast response times by using primary-key-only queries instead of full SQL execution at query time.

Proposed method

• Constructing a virtual document index from database rows, treating each row as a transient document for full-text indexing.
• Building a graph of foreign key relationships to model the logical schema structure of the database.
• Applying a novel join discovery algorithm to identify and traverse paths (trails) through the schema graph based on keyword queries.
• Presenting results as a tree-like navigational interface showing trails of related data across multiple tables.
• Using an inverted index with tf.idf-weighted postings to support efficient full-text search over database content.
• Implementing a lightweight, stateless interface using servlets and RPC (e.g., SOAP) to integrate with existing databases.

Experimental results

Research questions

• RQ1 How can keyword search be effectively supported in relational databases without requiring users to know SQL or table structures?
• RQ2 Can Memex-like trails be automatically discovered and presented in relational databases to improve navigation and information discovery?
• RQ3 What indexing and query techniques can ensure fast response times while maintaining semantic relevance in deep web database search?
• RQ4 How can foreign key relationships be leveraged to construct meaningful, user-understandable data trails across multiple tables?
• RQ5 What are the performance and scalability implications of indexing large relational databases for unstructured search?

Key findings

• The system successfully constructs Memex-like trails through foreign key relationships, enabling users to navigate complex database schemas using only keyword queries.
• By relying on primary-key-only queries at runtime, DbSurfer achieves fast response times, avoiding the performance overhead of full SQL execution.
• User studies on the UCL website showed that the combined search and navigation interface reduced time to find information, clicks, and increased user satisfaction compared to Google or Compass.
• The approach supports efficient indexing of diverse data formats (PDF, Office, Flash) by extracting textual content into a unified inverted index.
• The system demonstrates feasibility for integrating with existing databases via stored procedures and RPC protocols like SOAP.
• Security remains a challenge, as indexing through a single user account bypasses fine-grained access controls, risking exposure of sensitive data.