[Paper Review] Correlation and Substitution in SPARQL

Daniel Hernández, Claudio Gutiérrez+1 more|arXiv (Cornell University)|Jun 5, 2016

Semantic Web and Ontologies3 citations

TL;DR

This paper addresses ambiguities in SPARQL's treatment of variable correlation and substitution by proposing a formal semantics based on logic and programming language principles. It introduces a normalized 'bind' function to resolve inconsistencies in existing implementations, showing that Blazegraph/Fuseki (S1) and Virtuoso/rdf4j (S3) follow distinct but well-defined semantics, with only the latter two matching their respective formal models across all test cases.

ABSTRACT

In the current SPARQL specification the notion of correlation and substitution are not well defined. This problem triggers several ambiguities in the semantics. In fact, implementations as Fuseki and Virtuoso assume different semantics. In this technical report, we provide a semantics of correlation and substitution following the classic philosophy of substitution and correlation in logic, programming languages and SQL. We think this proposal not only fix the current ambiguities and problems, but helps to set a safe formal base to further extensions of the language. This work is part of an ongoing work of Daniel Hernandez. These anomalies in the W3C Specification of SPARQL 1.1 were detected early and reported no later than 2014, when two erratas were registered (cf. https://www.w3.org/2013/sparql-errata#errata-query-8 and https://www.w3.org/2013/sparql-errata#errata-query-10).

Motivation & Objective

• To resolve long-standing ambiguities in SPARQL's specification regarding variable correlation and substitution, which lead to inconsistent query results across implementations.
• To formalize three distinct semantics for variable visibility and substitution in SPARQL subqueries, based on how variables not projected in subqueries are treated.
• To provide a safe, formal foundation for future SPARQL extensions by aligning semantics with established principles from logic and programming languages.
• To evaluate and compare the behavior of major SPARQL engines (Fuseki, Blazegraph, Virtuoso, rdf4j) against the proposed formal semantics.
• To identify and explain discrepancies in engine behavior, particularly in cases involving unbound variables and nested subqueries.

Proposed method

• Proposes a new 'bind' function to replace the ill-defined 'substitute' function in the W3C SPARQL specification, ensuring consistent variable normalization.
• Introduces a normalization function 'norm(P)' that renames variables in a pattern P to ensure consistent role assignment across all occurrences.
• Defines three alternative semantics (S1, S2, S3) based on whether unprojected variables in subqueries are visible or not, with S1 and S3 matching real-world engine behaviors.
• Uses formal evaluation rules to compute query results under each semantics, comparing them against actual engine outputs on a set of 10 test queries.
• Analyzes edge cases involving blank nodes and unbound variables to ensure semantic safety and consistency.
• Validates the proposed semantics by comparing engine outputs against the formal models, identifying bugs and inconsistencies in implementations.

Experimental results

Research questions

• RQ1 How do current SPARQL implementations (Fuseki, Blazegraph, Virtuoso, rdf4j) differ in their handling of variable correlation and substitution in subqueries?
• RQ2 Why does the W3C SPARQL 1.1 specification lead to ambiguous semantics, and how can this be formally resolved?
• RQ3 What formal semantics can unify and clarify the divergent behaviors observed in existing SPARQL engines?
• RQ4 To what extent do real-world SPARQL engines conform to the proposed formal semantics, and where do they deviate?
• RQ5 How do blank nodes and unbound variables affect the correctness and safety of the proposed semantics?

Key findings

• Fuseki and Blazegraph follow semantics S1, where unprojected variables in subqueries are not visible, but Fuseki exhibits inconsistent behavior in queries 6 and 7, suggesting a bug.
• Virtuoso and rdf4j follow semantics S3, where unprojected variables are visible and can be substituted, and they match their formal model across all test cases.
• The proposed formal semantics S1 and S3 are logically consistent and safe, especially regarding blank nodes and variable scoping.
• The W3C's 'substitute' function is formally shown to be ill-defined and contradictory, necessitating replacement with the new 'bind' function.
• Query 8 and 10 reveal that some engines apply special handling to unbound variables, indicating potential implementation-specific interpretations.
• Only S3 (Virtuoso and rdf4j) correctly handles nested subqueries with correlated variables, while S1 (Fuseki/Blazegraph) fails in cases involving variable binding across multiple subqueries.