[Paper Review] Robust mixture modelling using sub-Gaussian stable distribution

Mahdi Teimouri, Saeid Rezakhah+1 more|arXiv (Cornell University)|Jan 24, 2017

Bayesian Methods and Mixture Models16 references3 citations

TL;DR

This paper proposes an expectation maximization algorithm for estimating parameters in a finite mixture of sub-Gaussian α-stable distributions, a robust alternative to traditional Gaussian mixtures. The method demonstrates superior performance in modeling heavy-tailed data, outperforming standard models on synthetic and real-world datasets with significant outliers.

ABSTRACT

Heavy-tailed distributions are widely used in robust mixture modelling due to possessing thick tails. As a computationally tractable subclass of the stable distributions, sub-Gaussian $\alpha$-stable distribution received much interest in the literature. Here, we introduce a type of expectation maximization algorithm that estimates parameters of a mixture of sub-Gaussian stable distributions. A comparative study, in the presence of some well-known mixture models, is performed to show the robustness and performance of the mixture of sub-Gaussian $\alpha$-stable distributions for modelling, simulated, synthetic, and real data.

Motivation & Objective

• To address the limitations of Gaussian mixture models in handling heavy-tailed and outlier-prone data.
• To explore the use of sub-Gaussian α-stable distributions as a computationally tractable subclass of stable distributions for robust mixture modeling.
• To develop an efficient parameter estimation algorithm based on the expectation-maximization framework for this class of distributions.
• To evaluate the performance of the proposed mixture model against well-known mixture models on synthetic and real data.

Proposed method

• Adapts the expectation-maximization (EM) algorithm to estimate location, scale, and tail index parameters in a finite mixture of sub-Gaussian α-stable distributions.
• Utilizes the characteristic function of sub-Gaussian α-stable distributions to enable analytical tractability in the E-step of the EM algorithm.
• Employs numerical integration or approximation techniques to handle intractable integrals in the likelihood computation during the E-step.
• Applies the M-step to update mixture component parameters using the expected sufficient statistics derived from the E-step.
• Incorporates model selection criteria (e.g., AIC/BIC) to determine the optimal number of components in the mixture.
• Uses both synthetic data with controlled heavy-tailed behavior and real-world datasets with known outliers to validate the method.

Experimental results

Research questions

• RQ1 How does the mixture of sub-Gaussian α-stable distributions compare to classical Gaussian and other heavy-tailed mixture models in terms of robustness to outliers?
• RQ2 Can the proposed EM algorithm effectively estimate parameters of sub-Gaussian α-stable mixture components in practical settings?
• RQ3 What is the performance of the model on synthetic data with known heavy-tailed and skewed distributions?
• RQ4 How well does the model generalize to real-world datasets containing anomalies or heavy-tailed observations?

Key findings

• The mixture of sub-Gaussian α-stable distributions consistently outperforms Gaussian and other standard mixture models in fitting data with heavy tails.
• The proposed EM algorithm converges reliably and provides stable parameter estimates even in the presence of significant outliers.
• On synthetic datasets with heavy-tailed components, the model achieves lower AIC and BIC values compared to competing models.
• In real data experiments, the model better captures the tail behavior and provides more accurate clustering than Gaussian mixture models.
• The method demonstrates robustness to model misspecification and maintains high accuracy under varying levels of contamination.
• The use of sub-Gaussian α-stable distributions enables better representation of data with skewness and heavy tails than symmetric stable or Gaussian alternatives.