[Paper Review] Refl-Spanners: A Purely Regular Approach to Non-Regular Core Spanners

Markus L. Schmid, Nicole Schweikardt|arXiv (Cornell University)|Oct 26, 2020

semigroups and automata theory8 citations

TL;DR

This paper introduces refl-spanners, a novel fragment of core spanners that embeds string-equality selections directly into regular expressions using ref-words, avoiding the high complexity of traditional core spanners. By treating equality constraints as part of the automaton’s language definition, refl-spanners achieve significantly better complexity bounds while retaining expressive power for most practical information extraction tasks.

ABSTRACT

The regular spanners (characterised by vset-automata) are closed under the algebraic operations of union, join and projection, and have desirable algorithmic properties. The core spanners (introduced by Fagin, Kimelfeld, Reiss, and Vansummeren (PODS 2013, JACM 2015) as a formalisation of the core functionality of the query language AQL used in IBM's SystemT) additionally need string-equality selections and it has been shown by Freydenberger and Holldack (ICDT 2016, Theory of Computing Systems 2018) that this leads to high complexity and even undecidability of the typical problems in static analysis and query evaluation. We propose an alternative approach to core spanners: by incorporating the string-equality selections directly into the regular language that represents the underlying regular spanner (instead of treating it as an algebraic operation on the table extracted by the regular spanner), we obtain a fragment of core spanners that, while having slightly weaker expressive power than the full class of core spanners, arguably still covers the intuitive applications of string-equality selections for information extraction and has much better upper complexity bounds of the typical problems in static analysis and query evaluation.

Motivation & Objective

• Address the high computational complexity of core spanners, particularly due to string-equality selections.
• Overcome undecidability and high evaluation complexity in static analysis and query evaluation for core spanners.
• Develop a fragment of core spanners that maintains practical expressiveness while enabling better algorithmic properties.
• Provide a purely regular alternative to core spanners that avoids the two-stage evaluation model of extracting then filtering span relations.
• Demonstrate that overlapping span equality selections are a primary source of complexity in core spanners.

Proposed method

• Introduce ref-words to represent documents paired with span-tuples that satisfy string-equality constraints, embedding equality checks directly into the regular language.
• Define reference-bounded refl-spanners as a fragment of core spanners where equality constraints are encoded via split operations on variables and non-overlapping constraints.
• Use the split operation $\textsf{split}_{\mathcal{X}}$ to decompose variables into multiple copies, enabling separate tracking of spans that must be equal.
• Apply the $\varsigma_E^{=}$ operator within a regular language framework via $\operatorname{\biguplus}_{\Lambda}$, which combines multiple regular languages under variable renaming.
• Leverage the fact that non-overlapping variables in split languages allow for regular representation, enabling construction of equivalent regular automata.
• Prove closure under projection and union by showing that the resulting language remains regular and can be represented by a finite automaton.

Experimental results

Research questions

• RQ1 Can string-equality selections in core spanners be integrated into the regular language definition to avoid post-processing filtering?
• RQ2 What is the impact of overlapping span equality constraints on the complexity of query evaluation and static analysis?
• RQ3 Is there a fragment of core spanners with lower complexity that still captures the essential functionality of AQL in IBM’s SystemT?
• RQ4 Can a purely regular formalism capture the core functionality of core spanners without sacrificing expressiveness for typical information extraction tasks?
• RQ5 Does embedding equality constraints into the automaton structure lead to better complexity bounds than treating them as post-filtering operations?

Key findings

• Refl-spanners achieve the same expressive power as core spanners with non-overlapping string-equality selections, but with significantly improved complexity bounds.
• The evaluation complexity of refl-spanners is bounded by the complexity of regular spanners, avoiding the undecidability and high complexity of core spanners.
• The class of reference-bounded refl-spanners is closed under union, natural join, projection, and span-fusion, with all operations preserving regularity.
• The construction of a refl-spanner for a given core spanner with non-overlapping equality constraints results in a variable set of size $O(|\mathcal{X}|^3)$, ensuring polynomial blow-up.
• The paper proves that $\varsigma_E^{=} \circ \llbracket L \rrbracket = \pi_\mathcal{Y} \biguplus_\Lambda \llbracket L' \rrbracket$ for a regular language $L'$, showing that equality constraints can be encoded in a regular automaton.
• The results demonstrate that overlapping span equality selections are a key source of complexity in core spanners, as refl-spanners avoid this by design.