[Paper Review] Towards a Polyglot Data Access Layer for a Low-Code Application Development Platform

Ana Nunes Alonso, João Abreu+5 more|arXiv (Cornell University)|Apr 28, 2020

Software System Performance and Reliability4 citations

TL;DR

This paper proposes a polyglot data access layer for the OutSystems low-code platform that enables seamless querying of diverse NoSQL data stores using an extended SQL-based query engine with embedded script snippets. By leveraging PostgreSQL with Foreign Data Wrappers and supporting schema introspection, query optimization, and native language overrides, the approach preserves low-code simplicity while enabling full exploitation of NoSQL capabilities, achieving efficient, scalable, and interactive data access without requiring NoSQL expertise.

ABSTRACT

Low-code application development as proposed by the OutSystems Platform enables fast mobile and desktop application development and deployment. It hinges on visual development of the interface and business logic but also on easy integration with data stores and services while delivering robust applications that scale. Data integration increasingly means accessing a variety of NoSQL stores. Unfortunately, the diversity of data and processing models, that make them useful in the first place, is difficult to reconcile with the simplification of abstractions exposed to developers in a low-code platform. Moreover, NoSQL data stores also rely on a variety of general purpose and custom scripting languages as their main interfaces. In this paper we propose a polyglot data access layer for the OutSystems Platform that uses SQL with optional embedded script snippets to bridge the gap between low-code and full access to NoSQL stores. In detail, we characterize the challenges for integrating a variety of NoSQL data stores; we describe the architecture and proof-of-concept implementation; and evaluate it with a sample application.

Motivation & Objective

• To address the challenge of integrating diverse NoSQL data stores into a low-code platform without requiring developers to master multiple NoSQL query languages.
• To preserve the low-code experience by abstracting NoSQL complexity while enabling full access to advanced query features when needed.
• To enable interactive, scalable, and secure data access to NoSQL stores through a unified, extensible query engine.
• To evaluate the feasibility and effectiveness of using extended SQL with embedded script fragments as a bridge between low-code abstractions and NoSQL system capabilities.

Proposed method

• The approach uses an extended relational model with nested composite types (maps, arrays) to represent diverse NoSQL data models with minimal conceptual overhead.
• A federated query engine based on PostgreSQL with Foreign Data Wrappers integrates multiple NoSQL sources, enabling cross-store queries and optimization.
• Schema introspection is performed automatically at integration time, with optional manual overrides for composite keys or custom structures.
• Native query fragments from NoSQL systems (e.g., MongoDB, Elasticsearch) are embedded within SQL queries to access advanced features not expressible in standard SQL.
• The system supports query pushdown and optimization, such as using $match in MongoDB pipelines, to improve performance and reduce data transfer.
• Materialized views are used to encapsulate complex, data-store-specific access paths, enabling reuse and optimization of native query logic.

Experimental results

Research questions

• RQ1 How can a low-code platform efficiently and transparently integrate with a wide variety of NoSQL data stores that have heterogeneous data and query models?
• RQ2 To what extent can an extended SQL-based query engine with embedded script fragments bridge the gap between low-code abstractions and full NoSQL functionality?
• RQ3 What role does schema introspection play in enabling developer productivity while maintaining performance and flexibility?
• RQ4 How effective is the combination of standard SQL with embedded native query fragments in supporting complex, optimized, and scalable data access?
• RQ5 What are the key architectural and implementation trade-offs in building a polyglot data access layer that supports both abstraction and low-level control?

Key findings

• The extended relational model with nested types successfully maps the majority of NoSQL data models with minimal conceptual or conversion overhead.
• Schema introspection significantly improves developer productivity, especially when combined with optional manual overrides for composite keys or custom structures.
• Embedding native query fragments in SQL allows developers to access advanced NoSQL features not expressible in standard SQL, without breaking the low-code abstraction.
• The use of a standard SQL engine with federated query capabilities and scripting layers proves highly effective for query optimization and composition, outperforming custom query engines in practice.
• Query pushdown optimizations, such as pushing down $match operations in MongoDB pipelines, significantly reduce data transfer and improve performance.
• Materialized view substitution is identified as a critical missing feature in the current prototype, highlighting its importance for encapsulating and reusing complex, data-store-specific access patterns.