[Paper Review] Apache VXQuery: A Scalable XQuery Implementation

E. Preston Carman, Till Westmann+3 more|arXiv (Cornell University)|Apr 1, 2015

Cloud Computing and Resource Management16 references9 citations

TL;DR

Apache VXQuery is a scalable, open-source XQuery processor built on Hyracks and Algebricks, enabling efficient parallel execution of XQuery workloads on large XML datasets. It achieves up to 3x faster performance than Saxon on single-node systems and 2.5x faster than Apache MRQL on cluster deployments, leveraging advanced optimization rules and dataflow-based parallelism for path expressions and joins.

ABSTRACT

The wide use of XML for document management and data exchange has created the need to query large repositories of XML data. To efficiently query such large data collections and take advantage of parallelism, we have implemented Apache VXQuery, an open-source scalable XQuery processor. The system builds upon two other open-source frameworks -- Hyracks, a parallel execution engine, and Algebricks, a language agnostic compiler toolbox. Apache VXQuery extends these two frameworks and provides an implementation of the XQuery specifics (data model, data-model dependent functions and optimizations, and a parser). We describe the architecture of Apache VXQuery, its integration with Hyracks and Algebricks, and the XQuery optimization rules applied to the query plan to improve path expression efficiency and to enable query parallelism. An experimental evaluation using a real 500GB dataset with various selection, aggregation and join XML queries shows that Apache VXQuery performs well both in terms of scale-up and speed-up. Our experiments show that it is about 3x faster than Saxon (an open-source and commercial XQuery processor) on a 4-core, single node implementation, and around 2.5x faster than Apache MRQL (a MapReduce-based parallel query processor) on an eight (4-core) node cluster.

Motivation & Objective

• To address the lack of scalable, parallel XQuery processors capable of efficiently querying large XML repositories.
• To build a high-performance XQuery processor that leverages modern dataflow execution engines for parallelism and optimization.
• To extend existing frameworks—Hyracks and Algebricks—with XQuery-specific data models, parsers, and optimization rules.
• To evaluate VXQuery’s performance on real-world, large-scale XML workloads (up to 500GB) in terms of speed-up and scale-up.
• To demonstrate that a dataflow-based approach outperforms MapReduce-based XQuery processors like Apache MRQL.

Proposed method

• Built VXQuery as an extension of Hyracks, a parallel dataflow execution engine, to support XQuery-specific execution plans.
• Integrated Algebricks, a language-agnostic compiler toolbox, to enable reusable, data-model-agnostic query optimization.
• Extended Algebricks with XQuery-specific rewrite rules to optimize path expressions and enable efficient parallel execution.
• Implemented a custom XQuery parser and XML data model to support XQuery 1.0 semantics within the Hyracks runtime.
• Used a hybrid hash join algorithm in VXQuery to reduce I/O and improve join performance compared to MapReduce-based approaches.
• Designed the system to hide parallelism details from users, allowing standard XQuery syntax while enabling automatic parallelization.

Experimental results

Research questions

• RQ1 Can a scalable XQuery processor be efficiently built by extending a general-purpose dataflow engine like Hyracks?
• RQ2 How does VXQuery’s performance compare to single-node XQuery processors like Saxon in terms of speed-up and scale-up?
• RQ3 To what extent does VXQuery outperform MapReduce-based XQuery systems like Apache MRQL on large XML workloads?
• RQ4 How effective are XQuery-specific optimization rules in improving path expression efficiency and enabling parallel execution?
• RQ5 Can VXQuery scale effectively across multiple nodes for large XML datasets, such as a 500GB weather data repository?

Key findings

• On a single 4-core node, VXQuery achieved approximately 3x speedup over SaxonHE, a leading open-source XQuery processor.
• On an 8-node cluster (4 cores per node), VXQuery was about 2.5x faster than Apache MRQL for the same XML queries.
• VXQuery demonstrated strong scale-up performance: with 7.2GB of data per node, query times remained stable as nodes were added, indicating efficient load distribution.
• With 66GB of data per node (total 528GB), VXQuery maintained high performance, showing effective horizontal scaling on very large XML datasets.
• The performance advantage over Apache MRQL stems from faster XML parsing (≈2x) and more efficient join processing via a hybrid hash join algorithm.
• Profiling revealed that VXQuery is CPU-bound during XML parsing, especially on large datasets, indicating that parsing optimization is a key area for future improvement.