[Paper Review] The Shapley Value of Inconsistency Measures for Functional Dependencies

Ester Livshits, Benny Kimelfeld|arXiv (Cornell University)|Sep 29, 2020

Logic, Reasoning, and Knowledge4 citations

TL;DR

This paper investigates the computational complexity of computing the Shapley value for functional dependency (FD) violations in inconsistent databases, proposing a systematic classification of inconsistency measures into tractable and intractable cases. It establishes that the number of violations and problematic facts are computable in polynomial time, while the drastic and minimum-cardinality repair measures exhibit a dichotomy based on the presence of an lhs chain in FDs, with approximation results provided for intractable cases.

ABSTRACT

Quantifying the inconsistency of a database is motivated by various goals including reliability estimation for new datasets and progress indication in data cleaning. Another goal is to attribute to individual tuples a level of responsibility to the overall inconsistency, and thereby prioritize tuples in the explanation or inspection of dirt. Therefore, inconsistency quantification and attribution have been a subject of much research in Knowledge Representation and, more recently, in Databases. As in many other fields, a conventional responsibility sharing mechanism is the Shapley value from cooperative game theory. In this paper, we carry out a systematic investigation of the complexity of the Shapley value in common inconsistency measures for functional-dependency (FD) violations. For several measures we establish a full classification of the FD sets into tractable and intractable classes with respect to Shapley-value computation. We also study the complexity of approximation in intractable cases.

Motivation & Objective

• To systematically analyze the computational complexity of Shapley value computation for inconsistency measures in databases with functional dependencies.
• To identify tractable and intractable classes of FD sets based on structural properties such as the presence of an lhs chain.
• To study the approximability of Shapley values for intractable inconsistency measures, including FPRAS and constant-ratio approximation.
• To extend the understanding of responsibility attribution in inconsistent databases using cooperative game theory.

Proposed method

• Models the inconsistency of a database as a cooperative game where tuples are players and inconsistency measures define the value of coalitions.
• Applies the Shapley value formula to quantify each tuple's marginal contribution to overall inconsistency across all possible coalitions.
• Uses combinatorial analysis and expectation computation over uniform random subsets to derive closed-form expressions for Shapley values.
• Employs dynamic programming and recursive decomposition for efficient computation in tractable cases, particularly for the number of violations and problematic facts.
• Derives bounds and approximation schemes (e.g., FPRAS) for intractable measures like the drastic and minimum-cardinality repair measures.
• Leverages structural properties of FD sets—especially the existence of an lhs chain (up to equivalence)—to classify complexity.

Figure 2 . The databases constructed in the reduction of the proof of Lemma 5 .

Experimental results

Research questions

• RQ1 Which inconsistency measures for functional dependencies allow polynomial-time computation of the Shapley value?
• RQ2 What structural property of FD sets (e.g., lhs chain) determines the tractability of Shapley value computation?
• RQ3 For intractable inconsistency measures, does a fully polynomial randomized approximation scheme (FPRAS) exist?
• RQ4 Can a constant-ratio approximation be achieved for the Shapley value of the cost of a cardinality repair in all FD sets?
• RQ5 How do the results generalize to other responsibility measures such as the Banzhaff Power Index or different constraint types?

Key findings

• The Shapley value for the number of violations ($\mathcal{I}_{\mathsf{MI}}$) and the number of problematic facts ($\mathcal{I}_{\mathsf{P}}$) can be computed in polynomial time.
• For the drastic measure ($\mathcal{I}_{\mathsf{d}}$), the Shapley value is tractable if and only if the FD set has an lhs chain (up to equivalence); otherwise, it is #P-hard.
• For the minimum-cardinality repair measure ($\mathcal{I}_{\mathsf{MC}}$), the Shapley value is tractable iff the FD set has an lhs chain, with a #P-hardness result otherwise.
• An FPRAS exists for the Shapley value of the drastic measure ($\mathcal{I}_{\mathsf{d}}$), but the existence of a constant-ratio approximation for the cost of a cardinality repair ($\mathcal{I}_{\mathsf{R}}$) remains open for certain FD sets.
• The tractable cases for $\mathcal{I}_{\mathsf{R}}$ are not fully characterized, particularly for FD sets without an lhs chain but still admitting polynomial-time repair computation.
• The paper identifies a gap in the complexity classification of $\mathcal{I}_{\mathsf{R}}$, especially for the bipartite matching constraint $\{A\rightarrow B, B\rightarrow A\}$, which has an FPRAS but is not yet fully classified.

Figure 4 . The data structure $T$ of our running example.