[Paper Review] PANDA: Query Evaluation in Submodular Width

Mahmoud Abo Khamis, Hung Q. Ngo+1 more|arXiv (Cornell University)|Feb 3, 2024

Algorithms and Data Compression4 citations

TL;DR

PANDA is a novel algorithm for conjunctive query evaluation that leverages proof-based translation of Shannon inequality derivations into database operations, achieving optimal runtime bounds. It computes full joins in time matching information-theoretic output size upper bounds and Boolean queries in time $ tilde{O}(2^{ extsf{subw}})$, where subw generalizes Marx's submodular width to include cardinality and degree constraints, significantly simplifying prior approaches.

ABSTRACT

In recent years, several information-theoretic upper bounds have been introduced on the output size and evaluation cost of database join queries. These bounds vary in their power depending on both the type of statistics on input relations and the query plans that they support. This motivated the search for algorithms that can compute the output of a join query in times that are bounded by the corresponding information-theoretic bounds. In this paper, we describe PANDA, an algorithm that takes a Shannon-inequality that underlies the bound, and translates each proof step into an algorithmic step corresponding to some database operation. PANDA computes answers to a conjunctive query in time given by the the submodular width plus the output size of the query. The version in this paper represents a significant simplification of the original version [ANS, PODS'17].

Motivation & Objective

• To design an efficient algorithm for conjunctive query evaluation that matches information-theoretic upper bounds on output size.
• To generalize Marx’s submodular width to incorporate both cardinality and degree constraints, including functional dependencies.
• To simplify and improve upon prior Worst-Case Optimal Join (WCOJ) algorithms by eliminating polynomial overhead in Boolean query evaluation.
• To unify the treatment of full and Boolean queries under a single framework grounded in entropic inequalities.
• To enable efficient evaluation of disjunctive rules using a novel decomposition strategy based on multiple tree decompositions.

Proposed method

• PANDA translates each step of a Shannon inequality proof into a corresponding database operation, directly linking proof structure to algorithmic steps.
• It uses a disjunctive rule abstraction to generalize conjunctive queries, allowing multiple atoms in the head and enabling efficient evaluation under complex statistics.
• The algorithm relies on a linear programming formulation of the submodular width, where the dual solution defines the runtime bound via $ extsf{subw} = m{w}^T m{n}$.
• It applies Farkas’ lemma to reframe the dual program into a standard linear program, enabling efficient computation of the witness to the inequality.
• The runtime is bounded by $ ilde{O}(2^{ extsf{subw}})$ for Boolean queries and by the information-theoretic output size upper bound for full joins.
• It generalizes the AGM bound to include degree constraints and functional dependencies through polymatroid constraints in the optimization framework.

Experimental results

Research questions

• RQ1 Can a proof-based approach to information-theoretic bounds be systematically translated into efficient query evaluation algorithms?
• RQ2 How can submodular width be generalized to incorporate degree constraints and functional dependencies while preserving tight runtime bounds?
• RQ3 Can the polynomial overhead in Marx’s original submodular width algorithm be eliminated to achieve exact $2^{ extsf{subw}}$ runtime?
• RQ4 What is the role of disjunctive rules in enabling efficient evaluation across multiple tree decompositions?
• RQ5 Can the framework be extended to non-Shannon inequalities for tighter bounds?

Key findings

• PANDA computes full conjunctive queries in time bounded by the information-theoretic upper bound on output size, incorporating both cardinality and degree constraints.
• For Boolean queries, PANDA achieves runtime $ ilde{O}(2^{ extsf{subw}})$, where $ extsf{subw}$ generalizes Marx’s submodular width to include richer input statistics.
• The algorithm simplifies prior WCOJ approaches by removing polynomial factors in the runtime, achieving tight asymptotic bounds.
• The submodular width $ extsf{subw}$ is defined as the optimal value of a linear program involving degree and cardinality constraints, with $ extsf{subw}_0 = m{w}^T m{n}$.
• The proof-based design ensures that each step in the Shannon inequality derivation corresponds directly to a database operation, making the algorithm both correct and efficient.
• The framework is extensible to $m{ u}$-norms of degree sequences, as shown in recent extensions to $ extsf{subw}$ with $m{ u}$-norms.