[Paper Review] Advances in exact Bayesian structure discovery in Bayesian networks

Mikko Koivisto|arXiv (Cornell University)|Jun 27, 2012

Bayesian Modeling and Causal Inference7 references17 citations

TL;DR

This paper presents a novel algorithm that computes the posterior probabilities for all potential edges in a Bayesian network structure in O(n 2^n) total time, achieving a significant O(n²) speedup over prior methods. By leveraging a forward-backward technique and fast M"obius transform algorithms, it enables efficient exact Bayesian structure discovery, allowing experimental study of moderate-size networks with up to 25 attributes and 10,000 records.

ABSTRACT

We consider a Bayesian method for learning the Bayesian network structure from complete data. Recently, Koivisto and Sood (2004) presented an algorithm that for any single edge computes its marginal posterior probability in O(n 2^n) time, where n is the number of attributes; the number of parents per attribute is bounded by a constant. In this paper we show that the posterior probabilities for all the n (n - 1) potential edges can be computed in O(n 2^n) total time. This result is achieved by a forward-backward technique and fast Moebius transform algorithms, which are of independent interest. The resulting speedup by a factor of about n^2 allows us to experimentally study the statistical power of learning moderate-size networks. We report results from a simulation study that covers data sets with 20 to 10,000 records over 5 to 25 discrete attributes

Motivation & Objective

• To enable efficient exact Bayesian structure learning in Bayesian networks with complete data.
• To overcome the computational bottleneck of computing edge posterior probabilities individually for each potential edge.
• To reduce the time complexity of computing all edge posteriors from O(n² 2^n) to O(n 2^n) for networks with bounded parent size.
• To support empirical analysis of statistical power in moderate-size networks (5–25 attributes) with realistic data sizes (20–10,000 records).

Proposed method

• Adopts a forward-backward technique to propagate local edge probabilities across the network structure.
• Employs fast M"obius transform algorithms to efficiently compute marginal posteriors over all possible edge configurations.
• Introduces a dynamic programming approach that reuses intermediate computations across edges to avoid redundant work.
• Bases the method on the assumption that the number of parents per node is bounded by a constant, ensuring scalability.
• Uses the joint posterior distribution over structures to compute edge-specific posterior probabilities efficiently.
• Optimizes the computation by exploiting the sparsity of the structure space and the modular nature of conditional independence.

Experimental results

Research questions

• RQ1 Can the computation of all edge posterior probabilities in a Bayesian network be performed in time significantly less than O(n² 2^n)?
• RQ2 Is it possible to compute the posterior probability of every potential edge in a Bayesian network in O(n 2^n) total time?
• RQ3 How does the proposed method scale in practice for networks with 5 to 25 discrete attributes and data sets of varying size?
• RQ4 What is the statistical power of exact Bayesian structure discovery in moderate-size networks under realistic data conditions?
• RQ5 Can the use of fast M"obius transforms and forward-backward propagation lead to practical improvements in structure learning?

Key findings

• The proposed algorithm computes the posterior probabilities for all n(n−1) potential edges in O(n 2^n) total time, representing a speedup by a factor of approximately n² over previous methods.
• The method enables experimental study of Bayesian structure discovery in networks with up to 25 attributes and 10,000 data records.
• The use of fast M"obius transforms and forward-backward propagation allows efficient reuse of intermediate computations across edges.
• The algorithm’s efficiency supports the analysis of statistical power in moderate-size networks, revealing insights into structure recovery under various data conditions.
• The implementation demonstrates practical feasibility of exact Bayesian structure learning beyond small networks, previously limited by computational cost.
• The results from the simulation study confirm that exact Bayesian methods can reliably recover true network structures even with moderate data sizes.