[Paper Review] Exact Inference in Networks with Discrete Children of Continuous Parents

Uri Lerner, Eran Segal+1 more|arXiv (Cornell University)|Jan 10, 2013

Bayesian Modeling and Causal Inference15 references66 citations

TL;DR

This paper presents the first exact inference algorithm for hybrid Bayesian networks with discrete children of continuous parents, extending Lauritzen's clique tree algorithm to handle conditional linear Gaussian models with discrete nodes dependent on continuous parents. The method computes exact distributions over discrete variables and exact first and second moments for continuous variables using numerical integration, achieving higher accuracy than prior approximate methods, especially with softmax CPDs.

ABSTRACT

Many real life domains contain a mixture of discrete and continuous variables and can be modeled as hybrid Bayesian Networks. Animportant subclass of hybrid BNs are conditional linear Gaussian (CLG) networks, where the conditional distribution of the continuous variables given an assignment to the discrete variables is a multivariate Gaussian. Lauritzen's extension to the clique tree algorithm can be used for exact inference in CLG networks. However, many domains also include discrete variables that depend on continuous ones, and CLG networks do not allow such dependencies to berepresented. No exact inference algorithm has been proposed for these enhanced CLG networks. In this paper, we generalize Lauritzen's algorithm, providing the first "exact" inference algorithm for augmented CLG networks - networks where continuous nodes are conditional linear Gaussians but that also allow discrete children ofcontinuous parents. Our algorithm is exact in the sense that it computes the exact distributions over the discrete nodes, and the exact first and second moments of the continuous ones, up to the accuracy obtained by numerical integration used within thealgorithm. When the discrete children are modeled with softmax CPDs (as is the case in many real world domains) the approximation of the continuous distributions using the first two moments is particularly accurate. Our algorithm is simple to implement and often comparable in its complexity to Lauritzen's algorithm. We show empirically that it achieves substantially higher accuracy than previous approximate algorithms.

Motivation & Objective

• To address the lack of exact inference algorithms for hybrid Bayesian networks where discrete variables depend on continuous parents.
• To extend Lauritzen's clique tree algorithm to support conditional linear Gaussian models with discrete children of continuous parents.
• To enable exact computation of distributions over discrete nodes and moments over continuous nodes in such networks.
• To improve accuracy over existing approximate inference methods in real-world domains with softmax CPDs.

Proposed method

• Extends Lauritzen's clique tree algorithm to handle hybrid conditional linear Gaussian networks with discrete children of continuous parents.
• Uses numerical integration to compute exact conditional distributions over discrete nodes given continuous parents.
• Maintains exact first and second moments for continuous variables through integration-based marginalization.
• Employs a factorization strategy that preserves conditional independence structures in the network.
• Supports softmax CPDs for discrete children, which are common in real-world applications.
• Integrates numerical quadrature within the clique tree framework to maintain computational tractability.

Experimental results

Research questions

• RQ1 Can exact inference be performed in hybrid Bayesian networks where discrete nodes have continuous parents?
• RQ2 How can the clique tree algorithm be modified to support discrete children of continuous parents while preserving exactness?
• RQ3 What is the impact of numerical integration accuracy on the quality of inference in such networks?
• RQ4 How does the proposed method compare in accuracy to existing approximate inference techniques?
• RQ5 To what extent does the use of softmax CPDs improve the approximation quality of continuous distributions?

Key findings

• The proposed algorithm achieves significantly higher accuracy than previous approximate inference methods in hybrid Bayesian networks with discrete children of continuous parents.
• Exact computation of discrete node distributions and continuous node moments is feasible using numerical integration within the clique tree framework.
• The method maintains computational complexity comparable to Lauritzen's original algorithm, making it practical for real-world use.
• When softmax CPDs are used, the approximation of continuous distributions using first and second moments is particularly accurate.
• Empirical results demonstrate that the algorithm outperforms existing approximate approaches in terms of distributional accuracy.