[Paper Review] On Pruning for Score-Based Bayesian Network Structure Learning

Alvaro H. C. Correia, James Cussens+1 more|TU/e Research Portal|May 23, 2019

Bayesian Modeling and Causal Inference18 references4 citations

TL;DR

This paper introduces tighter theoretical upper bounds for the Bayesian Dirichlet equivalent uniform (BDeu) score in Bayesian network structure learning, enabling more aggressive pruning of candidate parent sets without score computation. The proposed bounds, derived through novel mathematical analysis of the score function and maximum likelihood properties, are provably tighter than existing methods and significantly reduce the search space with minimal computational overhead, enhancing exact structure learning efficiency.

ABSTRACT

Many algorithms for score-based Bayesian network structure learning (BNSL), in particular exact ones, take as input a collection of potentially optimal parent sets for each variable in the data. Constructing such collections naively is computationally intensive since the number of parent sets grows exponentially with the number of variables. Thus, pruning techniques are not only desirable but essential. While good pruning rules exist for the Bayesian Information Criterion (BIC), current results for the Bayesian Dirichlet equivalent uniform (BDeu) score reduce the search space very modestly, hampering the use of the (often preferred) BDeu. We derive new non-trivial theoretical upper bounds for the BDeu score that considerably improve on the state-of-the-art. Since the new bounds are mathematically proven to be tighter than previous ones and at little extra computational cost, they are a promising addition to BNSL methods.

Motivation & Objective

• To address the computational bottleneck in exact Bayesian network structure learning (BNSL) caused by the exponential growth of candidate parent sets.
• To develop tighter theoretical upper bounds for the BDeu score that allow more effective pruning of suboptimal parent sets.
• To improve the scalability of exact BNSL with the BDeu score, which is often preferred but computationally challenging due to weak pruning in prior work.
• To provide bounds that are both mathematically provably tighter and computationally efficient for integration into existing BNSL algorithms.

Proposed method

• Derive a new upper bound, ub_g, by refining the mathematical structure of the BDeu score function, exploiting its properties to achieve tighter bounds than the prior state-of-the-art (ub_f).
• Introduce a second bound, ub_h, based on a modified maximum likelihood estimation approach, providing a complementary pruning mechanism with independent theoretical justification.
• Combine ub_g and ub_h via min{ub_g, ub_h} to create a tighter, hybrid bound that prunes disjoint regions of the search space.
• Ensure computational efficiency by designing bounds that require only a single pass through the data, making them linear-time and compatible with existing BNSL pipelines.
• Validate the bounds empirically on UCI datasets, demonstrating reduced number of computed scores and smaller candidate parent set lists.

Experimental results

Research questions

• RQ1 Can tighter theoretical upper bounds for the BDeu score be derived that significantly outperform the current state-of-the-art in pruning efficiency?
• RQ2 How do the new bounds ub_g and ub_h compare in their pruning effectiveness, and can they be combined to yield superior performance?
• RQ3 To what extent do the proposed bounds reduce the number of BDeu score computations and the size of candidate parent sets in practice?
• RQ4 Can the new bounds be computed efficiently enough to be practical in exact BNSL pipelines without increasing computational cost?

Key findings

• The proposed bound ub_g is mathematically proven to be tighter than the prior state-of-the-art bound ub_f for all candidate parent sets.
• The bound ub_h, derived from a new route based on maximum likelihood estimation, provides independent pruning power and is not dominated by ub_f.
• The combined bound min{ub_g, ub_h} achieves significantly tighter pruning than ub_f, as demonstrated in Figure 2, where it approaches the true optimal score more closely.
• Empirical results on UCI datasets (Figure 3) show that the new bounds drastically reduce the number of BDeu scores computed, especially for larger datasets with more than 10 variables.
• The bounds are computationally efficient, requiring only a single data pass, and can be integrated into any BNSL method with minimal overhead.