[Paper Review] Polyhedral aspects of score equivalence in Bayesian network structure learning

James Cussens, David Haws+1 more|arXiv (Cornell University)|Mar 3, 2015

Bayesian Modeling and Causal Inference29 references27 citations

TL;DR

This paper establishes a one-to-one correspondence between score-equivalent (SE) faces of the family-variable polytope and faces of the characteristic-imset polytope, showing that SE facets correspond to extreme supermodular functions. It proves that while non-SE facets can be safely omitted when maximizing SE objectives, SE facets alone are insufficient for exact optimization—highlighting the necessity of including constraints from the characteristic-imset polytope even when they do not define facets in the family-variable model.

ABSTRACT

This paper deals with faces and facets of the family-variable polytope and the characteristic-imset polytope, which are special polytopes used in integer linear programming approaches to statistically learn Bayesian network structure. A common form of linear objectives to be maximized in this area leads to the concept of score equivalence (SE), both for linear objectives and for faces of the family-variable polytope. We characterize the linear space of SE objectives and establish a one-to-one correspondence between SE faces of the family-variable polytope, the faces of the characteristic-imset polytope, and standardized supermodular functions. The characterization of SE facets in terms of extremality of the corresponding supermodular function gives an elegant method to verify whether an inequality is SE-facet-defining for the family-variable polytope. We also show that when maximizing an SE objective one can eliminate linear constraints of the family-variable polytope that correspond to non-SE facets. However, we show that solely considering SE facets is not enough as a counter-example shows; one has to consider the linear inequality constraints that correspond to facets of the characteristic-imset polytope despite the fact that they may not define facets in the family-variable mode.

Motivation & Objective

• To characterize score equivalence (SE) in the context of integer linear programming (ILP) for Bayesian network structure learning.
• To establish a one-to-one correspondence between SE faces of the family-variable polytope and faces of the characteristic-imset polytope.
• To determine whether SE facets alone are sufficient for exact optimization of SE objectives in the family-variable model.
• To clarify the role of constraints from the characteristic-imset polytope in ILP formulations, even when they do not define facets in the family-variable model.
• To provide a method for verifying whether an inequality is facet-defining for the family-variable polytope via extremality of the corresponding supermodular function.

Proposed method

• Define the family-variable polytope as the convex hull of DAG-codes (ηG) and the characteristic-imset polytope as the convex hull of characteristic imsets.
• Introduce the concept of score equivalence (SE) for linear objectives and faces of the family-variable polytope, based on invariance under Markov equivalence.
• Characterize the linear space of SE objectives and establish a correspondence between SE faces of the family-variable polytope and standardized supermodular set functions.
• Use the extremality of supermodular functions to characterize SE facets: an inequality is SE-facet-defining if and only if the corresponding supermodular function is extreme.
• Prove that generalized cluster inequalities are facet-defining for the family-variable polytope by showing their corresponding supermodular functions are extreme.
• Establish a one-to-one correspondence between SE faces of the family-variable polytope and faces of the characteristic-imset polytope, and derive the form of cluster inequalities in the characteristic-imset representation.

Experimental results

Research questions

• RQ1 Which linear objectives in Bayesian network structure learning are score equivalent, and how can they be characterized algebraically?
• RQ2 What is the precise relationship between faces of the family-variable polytope and faces of the characteristic-imset polytope under score equivalence?
• RQ3 Can all non-SE facets be safely removed when maximizing an SE objective in the family-variable model?
• RQ4 Are SE facets sufficient to define the feasible region for maximizing an SE objective in the family-variable polytope?
• RQ5 What is the role of constraints derived from the characteristic-imset polytope when they do not define facets in the family-variable model?

Key findings

• A one-to-one correspondence exists between SE faces of the family-variable polytope and faces of the characteristic-imset polytope, preserving inclusion relations.
• SE facets of the family-variable polytope correspond precisely to those facets of the characteristic-imset polytope that contain the 1-imset.
• An inequality is facet-defining for the family-variable polytope if and only if the corresponding supermodular function is extreme, providing an elegant verification method.
• Generalized cluster inequalities are proven to be facet-defining for the family-variable polytope via their corresponding extreme supermodular functions.
• A counterexample is constructed showing that even when all non-SE facets are removed and only SE facets are used, the optimal solution may still be infeasible—demonstrating that constraints from the characteristic-imset polytope are necessary.
• The vector η⋆ = (1+ε)·η†, which satisfies all non-negativity, modified convexity, and SE facet constraints but exceeds the optimal objective value of 16, proves that SE facets alone are insufficient for exact optimization.