[Paper Review] Learning Bayesian Networks: A Unification for Discrete and Gaussian Domains

David Heckerman, Dan Geiger|arXiv (Cornell University)|Feb 20, 2013

Bayesian Modeling and Causal Inference15 references125 citations

TL;DR

This paper unifies Bayesian network learning for discrete and Gaussian domains using a general Bayesian scoring metric derived from the Dirichlet and normal-Wishart distributions. It enables consistent structure learning across both domains by leveraging conjugate priors and marginal likelihoods, providing a theoretically grounded, unified framework for hybrid Bayesian network inference and scoring.

ABSTRACT

We examine Bayesian methods for learning Bayesian networks from a combination of prior knowledge and statistical data. In particular, we unify the approaches we presented at last year's conference for discrete and Gaussian domains. We derive a general Bayesian scoring metric, appropriate for both domains. We then use this metric in combination with well-known statistical facts about the Dirichlet and normal--Wishart distributions to derive our metrics for discrete and Gaussian domains.

Motivation & Objective

• To unify Bayesian network structure learning methods for discrete and Gaussian domains under a single probabilistic framework.
• To derive a general Bayesian scoring metric applicable to both discrete and continuous (Gaussian) variables.
• To leverage known statistical properties of the Dirichlet and normal-Wishart distributions to enable consistent scoring across domains.
• To provide a theoretically sound and computationally feasible approach for hybrid Bayesian network learning.
• To improve interpretability and consistency in structure learning by eliminating domain-specific scoring heuristics.

Proposed method

• Derives a general Bayesian scoring metric using marginal likelihoods under conjugate priors.
• Applies the Dirichlet distribution as a conjugate prior for multinomial (discrete) conditional probability tables.
• Applies the normal-Wishart distribution as a conjugate prior for multivariate Gaussian conditional distributions.
• Uses the marginal likelihood of the data under each network structure as the scoring function.
• Combines prior knowledge with observed data to compute posterior scores for network structures.
• Enables structure search via score-based optimization using the unified metric across mixed variable types.

Experimental results

Research questions

• RQ1 Can a single Bayesian scoring metric be derived that is valid for both discrete and Gaussian Bayesian networks?
• RQ2 How can conjugate priors (Dirichlet and normal-Wishart) be used to unify the scoring of discrete and continuous conditional distributions?
• RQ3 What is the theoretical basis for using marginal likelihood as a score in hybrid domains?
• RQ4 How does the unified metric ensure consistency and optimality in structure learning across mixed variable types?
• RQ5 What are the computational and statistical implications of applying this unified approach to real-world data?

Key findings

• A single Bayesian scoring metric is derived that applies uniformly to both discrete and Gaussian Bayesian networks.
• The use of conjugate priors (Dirichlet and normal-Wishart) enables exact computation of marginal likelihoods for both domains.
• The unified metric allows for consistent structure learning across hybrid domains without requiring separate scoring functions.
• The approach provides a theoretically grounded foundation for combining prior knowledge and statistical data in network learning.
• The method supports efficient score-based search over network structures using the same metric across mixed variable types.
• The framework enables principled learning of Bayesian networks with both discrete and continuous variables using a single, coherent scoring criterion.