[Paper Review] A Random Finite Set Model for Data Clustering

Dinh Phung, Ba‐Ngu Vo|arXiv (Cornell University)|Mar 14, 2017

Bayesian Methods and Mixture Models32 references3 citations

TL;DR

This paper proposes a Dirichlet Process mixture of Poisson Random Finite Sets (DP-RFS) for clustering set-valued data with an unknown number of clusters. By leveraging conjugate priors and a collapsed Gibbs sampler, the model automatically infers cluster count and parameters, successfully identifying highly unbalanced clusters where traditional methods fail.

ABSTRACT

The goal of data clustering is to partition data points into groups to minimize a given objective function. While most existing clustering algorithms treat each data point as vector, in many applications each datum is not a vector but a point pattern or a set of points. Moreover, many existing clustering methods require the user to specify the number of clusters, which is not available in advance. This paper proposes a new class of models for data clustering that addresses set-valued data as well as unknown number of clusters, using a Dirichlet Process mixture of Poisson random finite sets. We also develop an efficient Markov Chain Monte Carlo posterior inference technique that can learn the number of clusters and mixture parameters automatically from the data. Numerical studies are presented to demonstrate the salient features of this new model, in particular its capacity to discover extremely unbalanced clusters in data.

Motivation & Objective

• Address the challenge of clustering set-valued data, where each data point is a finite set of features rather than a fixed-dimensional vector.
• Overcome the limitation of existing clustering algorithms that require the number of clusters to be specified in advance.
• Develop a statistically principled model that can infer both the number of clusters and mixture parameters from data using nonparametric Bayesian methods.
• Enable robust clustering in imbalanced data scenarios where minority clusters are easily overlooked by standard algorithms.
• Provide a general framework applicable to diverse domains such as image analysis, text modeling, and spatial data, where data naturally appear as sets.

Proposed method

• Formalize data clustering as a random finite set (RFS) problem, modeling each data point as a realization of a Poisson RFS.
• Derive a conjugate prior for the Poisson RFS likelihood, enabling analytical marginalization over model parameters.
• Construct an infinite mixture model using a Dirichlet process prior on mixture weights to allow nonparametric inference over cluster count.
• Implement a collapsed Gibbs sampler that integrates out model parameters, improving convergence speed over full Gibbs sampling.
• Use predictive density and posterior predictive checks to guide cluster assignment and parameter estimation during MCMC sampling.
• Leverage the flexibility of Poisson RFS to model sparse, skewed, and unbalanced cluster structures, including rare or outlier clusters.

Experimental results

Research questions

• RQ1 Can a nonparametric Bayesian model based on random finite sets effectively cluster set-valued data without requiring the number of clusters to be pre-specified?
• RQ2 How does the proposed DP-RFS model perform in detecting extremely unbalanced clusters, especially when dominant clusters obscure minority ones?
• RQ3 To what extent does the conjugate prior structure for Poisson RFS likelihoods enable efficient and accurate posterior inference in a nonparametric mixture framework?
• RQ4 How does the collapsed Gibbs sampler in the DP-RFS model compare in convergence and accuracy to standard full Gibbs sampling in clustering tasks?
• RQ5 Can the DP-RFS model outperform state-of-the-art methods like infinite Gaussian Mixture Models (iGMM) in scenarios with sparse, imbalanced, or outlier-prone clusters?

Key findings

• The DP-RFS model successfully identified five clusters in a synthetic dataset with one dominant cluster (Poisson rate 100) and four sparse outlier clusters (rate 0.5), while iGMM missed the four minority clusters entirely.
• The model estimated the dominant cluster rate as 77.32 and the four outlier clusters at 0.34–0.38, closely matching the ground truth of 0.5.
• The collapsed Gibbs sampler converged faster than a standard full Gibbs sampler due to marginalization over model parameters.
• The DP-RFS model initially overestimated the number of clusters but converged toward the true count, demonstrating robustness to initial conditions.
• Unlike finite Mixture of Gaussians (MoG), which requires pre-specifying the number of clusters and fails to detect minority clusters, the DP-RFS model automatically inferred the correct cluster count.
• The model’s use of Poisson RFS likelihood provided sufficient flexibility to model skewed and sparse cluster structures, enabling detection of rare or anomalous patterns.