[Paper Review] Improved Acyclicity Reasoning for Bayesian Network Structure Learning with Constraint Programming

Fulya Trösser, Simon de Givry+1 more|arXiv (Cornell University)|Jun 23, 2021

Bayesian Modeling and Causal Inference21 references4 citations

TL;DR

This paper proposes enhanced acyclicity reasoning in constraint programming for Bayesian network structure learning (BNSL), introducing a polynomial-time cluster cut discovery algorithm, a greedy dual LP solver, and an optimized GAC propagator for the acyclicity constraint. Integrated into CPBayes as ELSA, these techniques reduce search time by orders of magnitude and outperform both GOBNILP and CPBayes on large BNSL instances, solving 9 more instances to optimality within time limits.

ABSTRACT

Bayesian networks are probabilistic graphical models with a wide range of application areas including gene regulatory networks inference, risk analysis and image processing. Learning the structure of a Bayesian network (BNSL) from discrete data is known to be an NP-hard task with a superexponential search space of directed acyclic graphs. In this work, we propose a new polynomial time algorithm for discovering a subset of all possible cluster cuts, a greedy algorithm for approximately solving the resulting linear program, and a generalised arc consistency algorithm for the acyclicity constraint. We embed these in the constraint programmingbased branch-and-bound solver CPBayes and show that, despite being suboptimal, they improve performance by orders of magnitude. The resulting solver also compares favourably with GOBNILP, a state-of-the-art solver for the BNSL problem which solves an NP-hard problem to discover each cut and solves the linear program exactly.

Motivation & Objective

• To address the NP-hard challenge of learning Bayesian network structures from discrete data using constraint programming.
• To improve the efficiency of acyclicity reasoning in CP-based solvers by reducing reliance on expensive, NP-hard cut generation.
• To develop lightweight yet effective inference mechanisms that balance solution quality and computational speed.
• To outperform existing solvers like GOBNILP and CPBayes by combining faster relaxation bounds with stronger pruning via GAC on acyclicity constraints.
• To scale BNSL solvers to larger instances by optimizing domain pruning and linear relaxation techniques.

Proposed method

• Proposes a polynomial-time algorithm to discover a subset of cluster cuts that provably improve the linear relaxation of the acyclicity constraint.
• Introduces a greedy algorithm for approximately solving the dual linear program, inspired by MaxSAT and WCSP methods, to accelerate relaxation computation.
• Designs a generalized arc consistency (GAC) propagator for the acyclicity constraint with improved time complexity and practical performance over prior work.
• Embeds these components into the CPBayes solver, forming a new solver called ELSA, which uses branch-and-bound with enhanced bounding and pruning.
• Employs a heuristic cluster ordering strategy to improve bound quality in the greedy dual LP solver, enhancing overall performance.
• Uses a hybrid approach combining symmetry breaking, subproblem caching, and pattern databases for lower bound estimation in the CP framework.

Experimental results

Research questions

• RQ1 Can a polynomial-time method for discovering acyclicity cuts improve the performance of CP-based BNSL solvers without solving NP-hard subproblems?
• RQ2 Does a greedy dual LP solver provide a sufficient approximation to the exact LP relaxation to significantly reduce runtime in BNSL?
• RQ3 Can an optimized GAC propagator for the acyclicity constraint reduce search space exploration more effectively than existing methods?
• RQ4 Does combining lightweight inference with strong pruning lead to better performance than state-of-the-art solvers like GOBNILP and CPBayes?
• RQ5 To what extent do cluster ordering and heuristic cut selection improve the quality of dual bounds in the context of BNSL?

Key findings

• ELSA, the enhanced CPBayes solver, solves 9 more BNSL instances to optimality within the time limit than GOBNILP, which was previously considered state-of-the-art.
• ELSA reduces search time by several orders of magnitude compared to CPBayes on most instances, with the exception of the bnetflix dataset.
• The GAC propagator reduces search time by up to 47% on larger instances, demonstrating that the computational cost is justified for larger problems.
• Cluster ordering improves bound quality and leads to 3 additional instances solved to optimality compared to using clusters in chronological order.
• The proposed polynomial-time cluster cut discovery algorithm provides a strong trade-off between inference strength and speed, outperforming GOBNILP’s NP-hard cut generation in practice.
• Despite using approximate dual solutions, the greedy LP solver significantly accelerates relaxation computation and enables faster pruning in the branch-and-bound search.