[Paper Review] Towards Polyglot Data Stores -- Overview and Open Research Questions

Daniel Glake, Felix Kiehn+3 more|arXiv (Cornell University)|Apr 12, 2022

Advanced Database Systems and Queries4 citations

TL;DR

This paper investigates polyglot data stores as a solution to the limitations of single-database systems in handling diverse, often conflicting, transactional and analytical workloads. It proposes a declarative, adaptive mediator that maps application requirements to optimal data store topologies, highlighting adaptivity as the key enabler for long-term system resilience and performance.

ABSTRACT

Nowadays, data-intensive applications face the problem of handling heterogeneous data with sometimes mutually exclusive use cases and soft non-functional goals such as consistency and availability. Since no single platform copes everything, various stores (RDBMS, NewSQL, NoSQL) for different workloads and use-cases have been developed. However, since each store is only a specialization, this motivates progress in polyglot data management emerged new systems called Mult- and Polystores. They are trying to access different stores transparently and combine their capabilities to achieve one or multiple given use-cases. This paper describes representative real-world use cases for data-intensive applications (OLTP and OLAP). It derives a set of requirements for polyglot data stores. Subsequently, we discuss the properties of selected Multi- and Polystores and evaluate them based on given needs illustrated by three common application use cases. We classify them into functional features, query processing technique, architecture and adaptivity and reveal a lack of capabilities, especially in changing conditions tightly integration. Finally, we outline the benefits and drawbacks of the surveyed systems and propose future research directions and current challenges in this area.

Motivation & Objective

• Address the challenge of managing heterogeneous data workloads in modern applications that require conflicting non-functional properties like consistency, availability, and performance.
• Identify the limitations of current polyglot and multistore systems, particularly their lack of runtime adaptivity and dynamic topology management.
• Propose a declarative, application-level specification model for data schemas annotated with functional and non-functional requirements (e.g., consistency levels, latency constraints).
• Design a smart, automated mediator that selects and routes data to appropriate data stores based on these annotations, aiming for optimal system topology.
• Enable autonomous adaptation to changing application and data requirements through runtime monitoring and topology reconfiguration.

Proposed method

• Define three real-world use cases—e-commerce (OLTP), analytics (OLAP), and hybrid OLTP/OLAP—to derive representative requirements for data management.
• Establish a framework of functional and non-functional features (e.g., consistency levels, query interfaces, schema flexibility) that a polyglot data store must support.
• Survey and compare existing polyglot and multistore systems (e.g., BigDAWG, Polypheny-DB) across architecture, query processing, and adaptivity dimensions.
• Propose a declarative schema annotation model allowing applications to specify functional and non-functional requirements (e.g., 'read-your-writes', 'low write latency').
• Design a smart ranking and routing algorithm that maps schema elements to appropriate data stores based on their capabilities and the annotated requirements.
• Integrate monitoring and feedback mechanisms to trigger adaptive changes in data store topology and routing policies at runtime.

Experimental results

Research questions

• RQ1 How can initial application requirements be mapped to an optimal initial topology of heterogeneous data stores?
• RQ2 How can a dynamic mapping between schema elements and data stores be maintained as requirements and data workloads evolve over time?
• RQ3 When and under what conditions should the system trigger topology adaptation—e.g., for performance, consistency, or availability—through data migration or store addition/removal?
• RQ4 How can the unique capabilities of individual data stores be fully exploited during query processing while preserving semantic completeness?
• RQ5 How can a polyglot system be monitored to enable autonomous, self-adaptive decisions regarding its underlying data store configuration?

Key findings

• Current polyglot and multistore systems exhibit a significant lack of adaptivity, relying on static configurations that fail to respond to changing workloads or requirements.
• Most surveyed systems do not fully exploit the unique capabilities of individual data stores, such as specialized indexing or consistency models, due to rigid integration patterns.
• The absence of runtime monitoring and feedback loops prevents systems from autonomously reconfiguring their topology in response to performance or availability changes.
• The paper identifies adaptivity as the central challenge and key enabler for future polyglot data stores, with performance gains potentially outweighing the overhead of dynamic reconfiguration.
• A declarative, annotation-based approach to schema specification enables precise expression of functional and non-functional requirements, forming a foundation for automated topology selection.
• The proposed mediator framework aims to close the gap between application-level SLAs and low-level data store capabilities through intelligent, automated mapping and routing.