[Paper Review] Mix-nets: Factored Mixtures of Gaussians in Bayesian Networks With Mixed Continuous And Discrete Variables

Scott Davies, Andrew Moore|arXiv (Cornell University)|Jan 16, 2013

Bayesian Modeling and Causal InferenceComputer Science34 references29 citations

TL;DR

This paper introduces mix-nets, a Bayesian network framework that models mixed continuous and discrete variables using factored mixtures of Gaussians, enabling efficient learning and inference without discretizing continuous variables. By combining fast, low-dimensional Gaussian mixture models via conditionalization, mix-nets achieve accurate joint density estimation and demonstrate strong performance on real and synthetic scientific datasets.

ABSTRACT

Recently developed techniques have made it possible to quickly learn accurate probability density functions from data in low-dimensional continuous space. In particular, mixtures of Gaussians can be fitted to data very quickly using an accelerated EM algorithm that employs multiresolution kd-trees (Moore, 1999). In this paper, we propose a kind of Bayesian networks in which low-dimensional mixtures of Gaussians over different subsets of the domain's variables are combined into a coherent joint probability model over the entire domain. The network is also capable of modeling complex dependencies between discrete variables and continuous variables without requiring discretization of the continuous variables. We present efficient heuristic algorithms for automatically learning these networks from data, and perform comparative experiments illustrated how well these networks model real scientific data and synthetic data. We also briefly discuss some possible improvements to the networks, as well as possible applications.

Motivation & Objective

• To develop a scalable method for learning Bayesian networks with mixed continuous and discrete variables.
• To enable accurate joint probability modeling without discretizing continuous variables.
• To leverage fast Gaussian mixture learning algorithms for efficient structure and parameter learning in high-dimensional domains.
• To support complex dependencies between continuous and discrete variables through conditionalization of factored mixtures.
• To demonstrate the feasibility and effectiveness of mix-nets on real scientific and synthetic datasets with dozens of variables.

Proposed method

• Proposes a Bayesian network structure where each node's conditional distribution is derived from a factored Gaussian mixture model over the node and its parents.
• Uses accelerated EM with multiresolution kd-trees to learn low-dimensional Gaussian mixtures quickly from data.
• Employs conditionalization: P(Xi|πi) = P(Xi,πi)/P(πi), where P(Xi,πi) is a learned mixture and P(πi) is its marginal, enabling consistent global modeling.
• Handles continuous variables via Gaussian mixture models, which are easily marginalized and conditionally reparameterized.
• Handles discrete variables via lookup tables or potential extensions to mixture models that jointly model discrete and continuous variables.
• Uses heuristic structure-learning algorithms to automatically construct the network from data, avoiding exhaustive search.

Experimental results

Research questions

• RQ1 Can factored mixtures of Gaussians be effectively combined into a coherent joint probability model for mixed-variable Bayesian networks?
• RQ2 How well can mix-nets model complex dependencies between continuous and discrete variables without discretization?
• RQ3 Can fast, accurate Gaussian mixture learning algorithms be adapted to learn Bayesian network parameters efficiently in high-dimensional settings?
• RQ4 What is the performance of mix-nets on real-world scientific datasets with mixed variable types?
• RQ5 How do mix-nets compare to alternative models such as single-Gaussian or kernel-based approaches in terms of accuracy and scalability?

Key findings

• Mix-nets achieve accurate joint density estimation on real scientific datasets with dozens of variables, demonstrating strong modeling capability.
• The method enables effective modeling of complex dependencies between continuous and discrete variables without discretization, preserving data fidelity.
• Experiments show mix-nets outperform baseline methods like single-Gaussian models in modeling accuracy on both synthetic and real datasets.
• The use of accelerated EM with kd-trees enables fast learning of low-dimensional mixtures, making the approach scalable to moderate-dimensional domains.
• Heuristic structure-learning algorithms successfully identify meaningful network structures from data, supporting practical deployment.
• The framework is extensible and supports future improvements such as variable grouping, alternative parameter learners, and hybrid discrete-continuous mixture models.