[Paper Review] Optimal Divide and Query (extended version)

David Insa, Josep Silva|arXiv (Cornell University)|Jul 2, 2011

Software Testing and Debugging Techniques15 references4 citations

TL;DR

This paper identifies fundamental flaws in the long-standing Divide and Query (D&Q) algorithm for algorithmic debugging, proving it suboptimal and incomplete. The authors propose a new optimal algorithm that uses weighted execution trees with individual node weights (including decimals) and introduces novel equations to guarantee identification of all optimal debugging solutions, significantly improving efficiency and correctness over prior methods.

ABSTRACT

Algorithmic debugging is a semi-automatic debugging technique that allows the programmer to precisely identify the location of bugs without the need to inspect the source code. The technique has been successfully adapted to all paradigms and mature implementations have been released for languages such as Haskell, Prolog or Java. During three decades, the algorithm introduced by Shapiro and later improved by Hirunkitti has been thought optimal. In this paper we first show that this algorithm is not optimal, and moreover, in some situations it is unable to find all possible solutions, thus it is incomplete. Then, we present a new version of the algorithm that is proven optimal, and we introduce some equations that allow the algorithm to identify all optimal solutions.

Motivation & Objective

• To identify and correct the theoretical and practical shortcomings of the established Divide and Query (D&Q) algorithm in algorithmic debugging.
• To prove that the original D&Q algorithm is not only suboptimal but also incomplete, failing to find all possible solutions in certain cases.
• To develop a new, formally proven optimal algorithm for algorithmic debugging that works with weighted execution trees where nodes can have non-uniform individual weights (including decimals).
• To introduce a set of mathematical equations that allow the algorithm to systematically identify all optimal debugging solutions based on node weights and structural properties.
• To generalize the debugging strategy beyond unit weights to support probabilistic or heuristic weight assignments reflecting the likelihood of bugs in different program components.

Proposed method

• The authors introduce a new algorithmic framework based on weighted execution trees (wET), where each node is assigned an individual weight representing its likelihood of being buggy.
• They define a new metric, the 'best node' criterion, using the formula $ w_{\text{Best}} - \frac{w_i_{\text{Best}}}{2} > \frac{w_n}{2} $, to guide node selection during debugging.
• The algorithm uses a recursive strategy that selects the heaviest child node of the current best node, with termination conditions based on the comparison of depth and upper bounds ($ d_n \geq u_n $).
• Key lemmas (e.g., Lemma 7, 8, 12, 13) are introduced to formally prove the correctness and optimality of node selection at each step.
• The algorithm dynamically evaluates node dominance using dominance relations ($ \gg $) and equivalence ($ \equiv $) to ensure no better node is missed.
• The method generalizes prior work by allowing non-integer weights and proving that optimal solutions are always found under the new strategy.

Experimental results

Research questions

• RQ1 Is the widely used Divide and Query algorithm truly optimal in terms of minimizing the number of questions in algorithmic debugging?
• RQ2 Can the original D&Q algorithm fail to find all possible optimal solutions, and if so, under what conditions?
• RQ3 What mathematical conditions must be satisfied to guarantee that a debugging strategy identifies all optimal solutions?
• RQ4 How can node weights (including decimal values) be used to improve the efficiency and accuracy of algorithmic debugging?
• RQ5 Can a new algorithm be formally proven optimal while supporting variable node weights and dynamic weight reassignment during debugging?

Key findings

• The original Divide and Query algorithm is proven suboptimal and incomplete, as it may fail to identify all optimal debugging solutions in certain execution tree configurations.
• The proposed new algorithm is formally proven optimal, guaranteeing termination at a node that is either strictly better than or equivalent to all other candidates in the search space.
• The algorithm correctly handles execution trees with non-unit node weights, including decimal and zero weights, enabling probabilistic or heuristic modeling of bug likelihood.
• The use of dominance relations ($ \gg $) and equivalence ($ \equiv $) ensures that no superior node is overlooked during the search process.
• The algorithm’s correctness is established through a series of lemmas and a formal inductive proof, showing that each iteration either improves the candidate or terminates at an optimal node.
• The new method outperforms existing strategies in both theoretical worst-case performance and practical scalability, as demonstrated by the authors’ experimental evaluation.