[Paper Review] Beyond Uniform Priors in Bayesian Network Structure Learning

Marco Scutari|arXiv (Cornell University)|Apr 12, 2017

Bayesian Modeling and Causal Inference25 references3 citations

TL;DR

This paper proposes the Bayesian Dirichlet sparse (BDs) marginal likelihood with a marginal uniform (MU) graph prior as a replacement for the standard uniform (U) + BDeu score in Bayesian network structure learning. It demonstrates that MU+BDs significantly improves structure learning accuracy on small and sparse datasets while maintaining competitive predictive performance, without increased computational cost.

ABSTRACT

Bayesian network structure learning is often performed in a Bayesian setting, evaluating candidate structures using their posterior probabilities for a given data set. Score-based algorithms then use those posterior probabilities as an objective function and return the maximum a posteriori network as the learned model. For discrete Bayesian networks, the canonical choice for a posterior score is the Bayesian Dirichlet equivalent uniform (BDeu) marginal likelihood with a uniform (U) graph prior, which assumes a uniform prior both on the network structures and on the parameters of the networks. In this paper, we investigate the problems arising from these assumptions, focusing on those caused by small sample sizes and sparse data. We then propose an alternative posterior score: the Bayesian Dirichlet sparse (BDs) marginal likelihood with a marginal uniform (MU) graph prior. Like U+BDeu, MU+BDs does not require any prior information on the probabilistic structure of the data and can be used as a replacement noninformative score. We study its theoretical properties and we evaluate its performance in an extensive simulation study, showing that MU+BDs is both more accurate than U+BDeu in learning the structure of the network and competitive in predicting power, while not being computationally more complex to estimate.

Motivation & Objective

• To address the limitations of the canonical U+BDeu score in Bayesian network structure learning, particularly under small sample sizes and sparse data.
• To investigate the theoretical and empirical shortcomings of uniform priors on both network structures and parameters in discrete Bayesian networks.
• To propose a new posterior score—BDs with MU prior—that better reflects sparse data characteristics while remaining noninformative and computationally efficient.
• To evaluate the new score's performance in terms of structural accuracy and predictive power across diverse benchmark networks and data regimes.

Proposed method

• Proposes the Bayesian Dirichlet sparse (BDs) marginal likelihood as a noninformative alternative to the BDeu score, which better handles sparse data by favoring sparse conditional probability tables.
• Introduces a marginal uniform (MU) graph prior that assigns uniform probability to all DAGs, preserving noninformative properties while avoiding the structural bias of the standard uniform prior.
• Derives theoretical properties of the MU+BDs score, including consistency and asymptotic behavior under sparse data conditions.
• Employs a comprehensive simulation study across 10 benchmark Bayesian networks (e.g., Alarm, Hailfinder, Hepar2) with varying sample-to-variable ratios (n/p = 0.1, 0.2, 0.5).
• Uses structural Hamming distance (SHD) to measure accuracy in recovering the true DAG and predictive log-likelihood to assess model performance.
• Compares MU+BDs against U+BDeu and other variants of the MU prior across multiple metrics, including average SHD and predictive log-likelihood.

Experimental results

Research questions

• RQ1 How do uniform priors on DAGs and parameters affect structure learning accuracy when data are small and sparse?
• RQ2 Does the BDs marginal likelihood with a MU graph prior yield more accurate network structures than U+BDeu under data scarcity?
• RQ3 Is the proposed MU+BDs score competitive in predictive performance compared to U+BDeu?
• RQ4 Does the new score maintain computational efficiency despite its improved theoretical properties?
• RQ5 How does the performance of MU+BDs vary across different network topologies and data regimes (e.g., n/p = 0.1 vs. 0.5)?

Key findings

• MU+BDs achieves significantly lower structural Hamming distance (SHD) than U+BDeu on all benchmark networks under small sample sizes (n/p = 0.1 and 0.2), indicating superior structure learning accuracy.
• On the Hailfinder network with n/p = 0.1, MU+BDs reduced SHD by 1.11 (from 5.31 to 4.20) compared to U+BDeu, with the best-performing MU variant (c=5) achieving 4.20 SHD.
• For the Hepar2 network at n/p = 0.1, MU+BDs reduced SHD from 3.73 (U+BDeu) to 3.41 (best MU variant), a 8.6% improvement.
• In predictive performance, MU+BDs consistently matches or slightly exceeds U+BDeu in log-likelihood, with the best variant (c=5) achieving 3.30 log-likelihood on Hepar2 at n/p = 0.5, compared to 3.31 for U+BDeu.
• The MU+BDs score maintains computational efficiency, with no increase in estimation complexity compared to U+BDeu, making it practical for large-scale applications.
• The simulation results show that MU+BDs is robust across diverse network structures, including both sparse (e.g., Water) and complex (e.g., Alarm) networks, across all data regimes tested.