[Paper Review] Query Answering under Matching Dependencies for Data Cleaning: Complexity and Algorithms

Jaffer Gardezi, Leopoldo Bertossi|arXiv (Cornell University)|Dec 27, 2011

Data Quality and Management30 references3 citations

TL;DR

This paper investigates query answering under matching dependencies (MDs) for data cleaning, proposing a systematic complexity analysis and a novel query rewriting methodology to efficiently compute resolved answers—those invariant across all minimally resolved instances (MRIs). The key contribution is a dichotomy result for MD sets and a tractable rewriting framework using positive Datalog with recursion and aggregation, enabling polynomial-time evaluation on certain classes of conjunctive queries and MDs.

ABSTRACT

Matching dependencies (MDs) have been recently introduced as declarative rules for entity resolution (ER), i.e. for identifying and resolving duplicates in relational instance $D$. A set of MDs can be used as the basis for a possibly non-deterministic mechanism that computes a duplicate-free instance from $D$. The possible results of this process are the clean, "minimally resolved instances" (MRIs). There might be several MRIs for $D$, and the "resolved answers" to a query are those that are shared by all the MRIs. We investigate the problem of computing resolved answers. We look at various sets of MDs, developing syntactic criteria for determining (in)tractability of the resolved answer problem, including a dichotomy result. For some tractable classes of MDs and conjunctive queries, we present a query rewriting methodology that can be used to retrieve the resolved answers. We also investigate connections with "consistent query answering", deriving further tractability results for MD-based ER.

Motivation & Objective

• To systematically analyze the computational complexity of computing resolved answers to conjunctive queries under matching dependencies (MDs).
• To identify syntactic criteria that distinguish tractable from intractable cases of resolved query answering under MDs.
• To develop a query rewriting methodology that retrieves resolved answers directly from the original dirty database instance without computing all MRIs.
• To establish connections between MD-based entity resolution and consistent query answering (CQA) under key constraints, leveraging existing CQA results.
• To extend tractable query classes beyond conjunctive queries using reductions to CQA and aggregation-based query rewriting.

Proposed method

• Proposes a chase-like procedure to compute minimally resolved instances (MRIs) from a dirty database using MDs, ensuring minimal changes through iterative application.
• Introduces the concept of resolved answers as those shared across all MRIs, avoiding the need to materialize all MRIs explicitly.
• Develops a query rewriting algorithm that transforms a conjunctive query into a new query expressible in positive Datalog with recursion and aggregation, which can be evaluated in polynomial time on the original dirty instance.
• Applies a dichotomy result to classify sets of two MDs as either in PTIME or NP-hard in data, based on syntactic properties.
• Establishes a reduction from MD-based resolved query answering to consistent query answering (CQA), using key constraints (KCs) as a foundation.
• Uses aggregation (COUNT) in first-order logic extensions to capture frequent values in groups of tuples with identical keys, enabling efficient rewriting for non-UJCQ queries.

Experimental results

Research questions

• RQ1 What are the syntactic conditions under which computing resolved answers under MDs is tractable or intractable?
• RQ2 Can a conjunctive query be rewritten into a new query that directly computes resolved answers from the original dirty database instance?
• RQ3 How do the complexity classes of resolved query answering under MDs relate to those in consistent query answering (CQA) under key constraints?
• RQ4 Can the query rewriting methodology be extended to queries outside the UJCQ class, and if so, under what conditions?
• RQ5 What is the role of aggregation and recursion in enabling efficient evaluation of resolved answers in polynomial time?

Key findings

• A dichotomy result is established for sets of two MDs: resolved answer computation is either in PTIME or NP-hard in data, based on syntactic criteria.
• For certain tractable classes of MDs and conjunctive queries, a polynomial-time computable query rewriting is possible using positive Datalog with recursion and aggregation.
• The rewritten query, while not always first-order, can be evaluated in polynomial time and returns exactly the resolved answers when posed to the original dirty instance.
• A connection is formally established between MD-based entity resolution and consistent query answering (CQA), enabling transfer of tractability results from CQA to MDs.
• The method extends beyond the UJCQ class: for example, a query not in UJCQ can still be rewritten using aggregation and CQA-based rewriting, as shown in Example 22.
• The use of aggregation (COUNT) in the rewritten query captures the most frequent values in groups of tuples with identical keys, enabling efficient resolution without explicit MRI computation.