[Paper Review] Efficiently Charting RDF

Oren Kalinsky, Oren Mishali+3 more|arXiv (Cornell University)|Nov 27, 2018

Data Quality and ManagementDecision Sciences72 references3 citations

TL;DR

This paper proposes Audit Join, a novel online aggregation algorithm that combines random walks from Wander Join with exact partial computations using Cache Trie Join to accelerate interactive visualization of large RDF knowledge graphs. It introduces an unbiased estimator for distinct counts and achieves significantly lower error than Wander Join across multiple knowledge graphs, enabling near real-time bar chart exploration with sub-minute convergence and under 1% mean error.

ABSTRACT

We propose a visual query language for interactively exploring large-scale knowledge graphs. Starting from an overview, the user explores bar charts through three interactions: class expansion, property expansion, and subject/object expansion. A major challenge faced is performance: a state-of-the-art SPARQL engine may require tens of minutes to compute the multiway join, grouping and counting required to render a bar chart. A promising alternative is to apply approximation through online aggregation, trading precision for performance. However, state-of-the-art online aggregation algorithms such as Wander Join have two limitations for our exploration scenario: (1) a high number of rejected paths slows the convergence of the count estimations, and (2) no unbiased estimator exists for counts under the distinct operator. We thus devise a specialized algorithm for online aggregation that augments Wander Join with exact partial computations to reduce the number of rejected paths encountered, as well as a novel estimator that we prove to be unbiased in the case of the distinct operator. In an experimental study with random interactions exploring two large-scale knowledge graphs, our algorithm shows a clear reduction in error with respect to computation time versus Wander Join.

Motivation & Objective

• To enable interactive, real-time exploration of large-scale RDF knowledge graphs through a visual query language based on Shneiderman’s 'overview first, zoom and filter, then details-on-demand' principle.
• To address the performance bottleneck in rendering bar charts from SPARQL queries over massive knowledge graphs, where exact computation takes minutes to hours.
• To design an online aggregation algorithm that reduces estimation error compared to state-of-the-art methods like Wander Join, particularly under high selectivity and distinct aggregations.
• To develop an unbiased estimator for count aggregates under the DISTINCT operator, which is critical for accurate visualization in knowledge graph exploration.
• To integrate exact computation (via Cache Trie Join) selectively during online aggregation to improve convergence speed and accuracy without sacrificing performance.

Proposed method

• Audit Join combines the random walk mechanism of Wander Join with selective exact computation using the Cache Trie Join algorithm to reduce the number of rejected paths and improve convergence.
• At each step, the algorithm estimates selectivity and switches from random walk to exact computation for the remaining suffix space when the estimated cost is lower.
• A novel, provably unbiased estimator is introduced for count aggregates under the DISTINCT operator, addressing a key limitation of existing online aggregation methods.
• The algorithm dynamically updates the estimator during execution, ensuring that the expected value of the estimate equals the true count, even under distinct aggregation.
• The framework supports three user interactions—class expansion, property expansion, and subject/object expansion—each generating a new bar chart in an interactive exploration pipeline.
• The system is evaluated using randomly generated exploration queries over DBpedia and LinkedGeoData, comparing error versus computation time against Wander Join and exact execution.

Experimental results

Research questions

• RQ1 Can online aggregation with selective exact computation reduce estimation error in RDF bar chart visualization compared to pure online aggregation?
• RQ2 Does the proposed unbiased estimator for distinct counts improve accuracy in knowledge graph exploration workloads?
• RQ3 How does the integration of exact partial computation via Cache Trie Join affect convergence speed and error reduction in interactive RDF exploration?
• RQ4 To what extent does Audit Join outperform Wander Join in terms of error reduction and runtime efficiency across different exploration depths and knowledge graph sizes?
• RQ5 Is the performance benefit of Audit Join consistent across diverse knowledge graphs, including those with high selectivity and duplicate-rich queries?

Key findings

• Audit Join reduces estimation error by orders of magnitude compared to Wander Join, achieving less than 1% mean error in most cases within sub-minute timeframes.
• The algorithm converges significantly faster than both pure online aggregation (Wander Join) and exact execution (Virtuoso and Cache Trie Join), making interactive exploration feasible.
• For the non-distinct case, Audit Join still outperforms Wander Join by a substantial margin, demonstrating that the benefit of partial exact computation extends beyond the distinct operator.
• The number of rejected paths in Wander Join increases with query selectivity and complexity, slowing convergence; Audit Join mitigates this by replacing high-rejection segments with exact computation.
• The relative performance improvement of Audit Join over Wander Join increases with the number of exploration steps, indicating growing advantage in deeper, more complex queries.
• Exact computation via Cache Trie Join reduces runtime from hours (Virtuoso) to tens of seconds, but remains incompatible with interactive use; online aggregation with Audit Join closes this gap effectively.