[Paper Review] Finding Optimal Bayesian Networks

David Maxwell Chickering, Christopher Meek|arXiv (Cornell University)|Dec 12, 2012

Bayesian Modeling and Causal Inference10 references112 citations

TL;DR

This paper establishes that greedy Bayesian network search algorithms using asymptotically consistent scoring criteria converge to an inclusion-optimal Bayesian network structure under the weaker composition property assumption, rather than requiring the stronger perfectness condition. The key contribution is proving that such algorithms identify a model that contains the true generative distribution and has no smaller sub-model that does so, even when unobserved variables or selection bias are present.

ABSTRACT

In this paper, we derive optimality results for greedy Bayesian-network search algorithms that perform single-edge modifications at each step and use asymptotically consistent scoring criteria. Our results extend those of Meek (1997) and Chickering (2002), who demonstrate that in the limit of large datasets, if the generative distribution is perfect with respect to a DAG defined over the observable variables, such search algorithms will identify this optimal (i.e. generative) DAG model. We relax their assumption about the generative distribution, and assume only that this distribution satisfies the {em composition property} over the observable variables, which is a more realistic assumption for real domains. Under this assumption, we guarantee that the search algorithms identify an {em inclusion-optimal} model; that is, a model that (1) contains the generative distribution and (2) has no sub-model that contains this distribution. In addition, we show that the composition property is guaranteed to hold whenever the dependence relationships in the generative distribution can be characterized by paths between singleton elements in some generative graphical model (e.g. a DAG, a chain graph, or a Markov network) even when the generative model includes unobserved variables, and even when the observed data is subject to selection bias.

Motivation & Objective

• To relax the strong assumption of perfectness in Bayesian network structure learning.
• To establish conditions under which greedy search algorithms converge to an inclusion-optimal model.
• To demonstrate that the composition property ensures convergence to a model that contains the true generative distribution.
• To show that the composition property holds even when unobserved variables or selection bias affect observed data.
• To extend prior results on asymptotic consistency of scoring criteria to more realistic data-generating processes.

Proposed method

• The authors define the composition property as a necessary and sufficient condition for inclusion-optimality in Bayesian network structure learning.
• They analyze greedy search algorithms that modify one edge at a time and use asymptotically consistent scoring criteria.
• The method relies on proving that under the composition property, any local optimum corresponds to an inclusion-optimal model.
• The proof technique involves analyzing the structure of dependence relationships in graphical models, including DAGs, chain graphs, and Markov networks.
• The approach generalizes prior results by Meek (1997) and Chickering (2002) by replacing the perfectness assumption with the composition property.
• The framework applies to models with unobserved variables and data subject to selection bias, as long as the composition property holds.

Experimental results

Research questions

• RQ1 Under what conditions do greedy Bayesian network search algorithms converge to a model that contains the true generative distribution?
• RQ2 Can the convergence to an optimal model be guaranteed without assuming the generative distribution is perfect with respect to a DAG?
• RQ3 Does the composition property hold in the presence of unobserved variables or selection bias in observed data?
• RQ4 Is there a weaker condition than perfectness that still ensures convergence to an inclusion-optimal model?
• RQ5 Can asymptotically consistent scoring criteria be used to identify inclusion-optimal structures under the composition property?

Key findings

• Greedy search algorithms using asymptotically consistent scoring criteria converge to an inclusion-optimal Bayesian network structure under the composition property.
• The composition property is satisfied whenever dependence relationships in the generative distribution can be represented by paths in a graphical model, even with unobserved variables.
• The composition property holds even when observed data is subject to selection bias.
• The inclusion-optimal model identified by the algorithm contains the true generative distribution and has no smaller sub-model that does.
• The results generalize prior work by relaxing the perfectness assumption, making the theoretical guarantees applicable to a broader class of real-world domains.
• The framework supports learning from data with latent confounders or selection bias, provided the composition property is met.