[Paper Review] Efficient Learning of Bounded-Treewidth Bayesian Networks from Complete and Incomplete Data Sets

Mauro Scanagatta, Giorgio Corani+3 more|arXiv (Cornell University)|Feb 7, 2018

Bayesian Modeling and Causal Inference21 references3 citations

TL;DR

This paper introduces k-MAX, an anytime algorithm for learning bounded-treewidth Bayesian networks from complete and incomplete data, significantly outperforming existing methods in structure score and scalability. When integrated into structural EM, k-MAX enables efficient, linear-time data imputation at ~10x faster speed than state-of-the-art random forest methods, with comparable accuracy and full parallelizability.

ABSTRACT

Learning a Bayesian networks with bounded treewidth is important for reducing the complexity of the inferences. We present a novel anytime algorithm (k-MAX) method for this task, which scales up to thousands of variables. Through extensive experiments we show that it consistently yields higher-scoring structures than its competitors on complete data sets. We then consider the problem of structure learning from incomplete data sets. This can be addressed by structural EM, which however is computationally very demanding. We thus adopt the novel k-MAX algorithm in the maximization step of structural EM, obtaining an efficient computation of the expected sufficient statistics. We test the resulting structural EM method on the task of imputing missing data, comparing it against the state-of-the-art approach based on random forests. Our approach achieves the same imputation accuracy of the competitors, but in about one tenth of the time. Furthermore we show that it has worst-case complexity linear in the input size, and that it is easily parallelizable.

Motivation & Objective

• To develop a scalable, high-performance algorithm for learning Bayesian networks with bounded treewidth from large-scale data.
• To address the computational bottleneck of structure learning from incomplete data by integrating k-MAX into structural EM.
• To enable efficient data imputation with bounded-treewidth networks while maintaining high accuracy and low computational cost.
• To achieve linear worst-case time complexity and native parallelizability for large-scale applications.

Proposed method

• k-MAX employs advanced heuristics to iteratively optimize the structure of Bayesian networks under a treewidth constraint, ensuring high-scoring DAGs.
• The algorithm is anytime, allowing early stopping with a current best solution, and is designed for scalability to thousands of variables.
• k-MAX is integrated into the maximization step of structural EM (SEM), enabling efficient computation of expected sufficient statistics via bounded-treewidth inference.
• The method performs data imputation by exact inference on learned bounded-treewidth networks, with each query costing O(n) time when treewidth and variable cardinality are constant.
• The entire pipeline is parallelizable: parent set identification, k-MAX optimization, and expectation-phase queries can be distributed across cores.
• Parameter tuning allows users to balance imputation accuracy and runtime by adjusting time limits per algorithmic phase.

Experimental results

Research questions

• RQ1 Can k-MAX outperform existing anytime algorithms like k-greedy in learning high-scoring bounded-treewidth Bayesian networks on complete data?
• RQ2 Can k-MAX be efficiently integrated into structural EM to scale to thousands of variables for incomplete data?
• RQ3 Does the resulting SEM-k-MAX method achieve competitive imputation accuracy while significantly reducing runtime compared to state-of-the-art approaches?
• RQ4 What is the theoretical time complexity of SEM-k-MAX, and does it scale linearly with input size?
• RQ5 To what extent can the SEM-k-MAX pipeline be parallelized across multiple processors?

Key findings

• k-MAX consistently achieves higher-scoring Bayesian network structures than k-greedy and other competitors on complete data sets, especially in high-dimensional settings.
• SEM-k-MAX achieves the same data imputation accuracy as state-of-the-art random forest-based methods but runs approximately ten times faster.
• The worst-case time complexity of SEM-k-MAX is linear in the input size (O(nd)), making it scalable to massive datasets with thousands of variables.
• The method is natively parallelizable, with independent execution possible across variables, cores, and data points, enabling efficient distributed deployment.
• Parameter tuning via execution time limits (e.g., t=1,5,10) allows users to balance accuracy and speed, with t=1 offering the best trade-off in experiments.
• The approach is the first in the literature to combine efficient SEM with bounded-treewidth learning at scale, enabling tractable inference on incomplete data.