[Paper Review] Enhancing Invenio Digital Library With An External Relevance Ranking Engine

Patrick Glauner|arXiv (Cornell University)|Nov 4, 2012

Information Retrieval and Data Mining3 citations

TL;DR

This paper enhances Invenio, a digital library system developed at CERN, by integrating external relevance ranking engines—specifically Solr and Xapian—via a system-agnostic bridge to improve information retrieval and word similarity ranking. The integration boosts search relevance and scalability, validated through performance testing, offering a modular, extensible solution for digital library systems.

ABSTRACT

Invenio is a comprehensive web-based free digital library software suite originally developed at CERN. In order to improve its information retrieval and word similarity ranking capabilities, the goal of this thesis is to enhance Invenio by bridging it with modern external information retrieval systems. In the first part a comparison of various information retrieval systems such as Solr and Xapian is made. In the second part a system-independent bridge for word similarity ranking is designed and implemented. Subsequently, Solr and Xapian are integrated in Invenio via adapters to the bridge. In the third part scalability tests are performed. Finally, a future outlook is briefly discussed.

Motivation & Objective

• To improve Invenio’s information retrieval and word similarity ranking capabilities beyond its native functionality.
• To evaluate and compare modern external IR systems such as Solr and Xapian for integration with Invenio.
• To design and implement a system-independent bridge enabling interoperability between Invenio and external ranking engines.
• To assess the scalability and performance of the enhanced Invenio system with external engines.
• To provide a modular, extensible architecture for future integration of additional IR systems.

Proposed method

• A comparative analysis of information retrieval systems, including Solr and Xapian, was conducted to assess suitability for Invenio integration.
• A system-agnostic bridge was designed to abstract differences between external IR engines, enabling uniform interaction with Invenio.
• Adapters were implemented to connect Invenio with Solr and Xapian through the bridge, enabling external ranking functionality.
• The system was extended to support relevance ranking and word similarity queries using external engines instead of Invenio’s native mechanisms.
• Scalability tests were performed to evaluate performance under increasing data and query loads.
• The architecture was designed to allow future pluggable integration of additional IR systems without modifying core Invenio components.

Experimental results

Research questions

• RQ1 How can external relevance ranking engines be effectively integrated into the Invenio digital library system?
• RQ2 What are the performance and scalability characteristics of integrating Solr and Xapian with Invenio?
• RQ3 Can a system-agnostic bridge be designed to decouple Invenio from specific IR engines while maintaining functionality?
• RQ4 How does the use of external engines improve word similarity and relevance ranking compared to Invenio’s native capabilities?
• RQ5 What are the architectural and implementation challenges in extending Invenio with external IR systems?

Key findings

• The integration of Solr and Xapian via the bridge significantly improved Invenio’s relevance ranking and word similarity capabilities.
• The system-agnostic bridge successfully abstracted differences between IR engines, enabling seamless pluggable integration.
• Scalability tests demonstrated that the enhanced system maintained acceptable performance under increasing query and data loads.
• The modular architecture allows for future extension with additional IR systems without modifying core Invenio components.
• The solution provides a practical, extensible path to enhance search quality in digital library systems using mature external IR technologies.
• The approach enables Invenio to leverage state-of-the-art indexing and ranking features from external systems while preserving system modularity.