[Paper Review] Model-based clustering of Gaussian copulas for mixed data

Matthieu Marbac, Christophe Biernacki+1 more|arXiv (Cornell University)|May 6, 2014

Bayesian Methods and Mixture Models35 references4 citations

TL;DR

This paper proposes a Gaussian copula mixture model for clustering mixed-type data (continuous, integer, and ordinal variables), leveraging copulas to separately model marginal distributions and intra-component correlations. The method enables interpretable clustering with meaningful visualization via PCA on latent variables, outperforming independent-component models by reducing bias and component count while maintaining flexibility across diverse data types.

ABSTRACT

Clustering task of mixed data is a challenging problem. In a probabilistic framework, the main difficulty is due to a shortage of conventional distributions for such data. In this paper, we propose to achieve the mixed data clustering with a Gaussian copula mixture model, since copulas, and in particular the Gaussian ones, are powerful tools for easily modelling the distribution of multivariate variables. Indeed, considering a mixing of continuous, integer and ordinal variables (thus all having a cumulative distribution function), this copula mixture model defines intra-component dependencies similar to a Gaussian mixture, so with classical correlation meaning. Simultaneously, it preserves standard margins associated to continuous, integer and ordered features, namely the Gaussian, the Poisson and the ordered multinomial distributions. As an interesting by-product, the proposed mixture model generalizes many well-known ones and also provides tools of visualization based on the parameters. At a practical level, the Bayesian inference is retained and it is achieved with a Metropolis-within-Gibbs sampler. Experiments on simulated and real data sets finally illustrate the expected advantages of the proposed model for mixed data: flexible and meaningful parametrization combined with visualization features.

Motivation & Objective

• To address the challenge of clustering mixed-type data (continuous, integer, ordinal) using a probabilistic framework.
• To overcome the lack of conventional multivariate distributions for mixed data by leveraging Gaussian copulas.
• To ensure interpretable component parameters by preserving standard marginal distributions (Gaussian, Poisson, ordered multinomial).
• To provide a visualization tool based on model parameters for enhanced interpretability of clusters.
• To develop a robust, Bayesian inference framework that handles mixed data with dependencies and missing values.

Proposed method

• Model each cluster using a Gaussian copula, which decouples marginal distributions from dependence structure.
• Preserve standard margins: Gaussian for continuous, Poisson for integer, and ordered multinomial for ordinal variables.
• Use latent variables from the copula to enable PCA-like visualization of individuals within each component.
• Apply a Metropolis-within-Gibbs sampler for Bayesian inference, simplifying estimation in the presence of mixed margins.
• Introduce an homoscedastic variant to reduce parameter count by assuming equal correlation matrices across components.
• Adapt the Gibbs sampler for handling missing data under missing-at-random assumptions.

Experimental results

Research questions

• RQ1 Can a Gaussian copula mixture model effectively cluster mixed-type data while preserving interpretable marginal distributions?
• RQ2 How does the proposed model compare to the locally independent mixture model in terms of bias and component count?
• RQ3 Can the latent structure of the copula enable meaningful visualization of cluster-specific patterns?
• RQ4 To what extent does the model generalize existing finite mixture models for homogeneous data?
• RQ5 How robust is the model when fitted to data generated from alternative models?

Key findings

• The model successfully clusters mixed data with three distinct fire behavior classes: unpredictable fires (9% of data), predictable summer fires (78%), and winter fires (13%), each with interpretable marginal and correlation structures.
• The Gaussian copula mixture model reduced the number of components needed compared to the locally independent model, indicating lower bias and improved model fit.
• Visualization via PCA on latent variables clearly separated component 3 (winter fires) from the others, highlighting its distinct distributional characteristics.
• The correlation matrices revealed meaningful dependencies, such as between FFMC and DMC values in winter fires and between high temperature and summer fire conditions.
• The model demonstrated robustness by fitting data generated from a different model, suggesting generalization capability.
• The homoscedastic variant reduced parameter count without sacrificing model performance, offering a more parsimonious alternative.