[Paper Review] Initialization Free Graph Based Clustering

L. Galluccio, Olivier Michel+3 more|ArXiv.org|Sep 24, 2009

Remote-Sensing Image Classification47 references3 citations

TL;DR

This paper proposes an initialization-free graph-based clustering method that estimates the number of clusters and their centroids using a minimal spanning tree (MST) constructed via Prim’s algorithm. By modeling vertex distribution as Poisson and thresholding the MST's edge-length trajectory, it detects cluster modes and initializes K-means without prior knowledge, significantly improving performance on astrophysical and hyperspectral image data using information-theoretic divergences like Kullback-Leibler divergence.

ABSTRACT

This paper proposes an original approach to cluster multi-component data sets, including an estimation of the number of clusters. From the construction of a minimal spanning tree with Prim's algorithm, and the assumption that the vertices are approximately distributed according to a Poisson distribution, the number of clusters is estimated by thresholding the Prim's trajectory. The corresponding cluster centroids are then computed in order to initialize the generalized Lloyd's algorithm, also known as $K$-means, which allows to circumvent initialization problems. Some results are derived for evaluating the false positive rate of our cluster detection algorithm, with the help of approximations relevant in Euclidean spaces. Metrics used for measuring similarity between multi-dimensional data points are based on symmetrical divergences. The use of these informational divergences together with the proposed method leads to better results, compared to other clustering methods for the problem of astrophysical data processing. Some applications of this method in the multi/hyper-spectral imagery domain to a satellite view of Paris and to an image of the Mars planet are also presented. In order to demonstrate the usefulness of divergences in our problem, the method with informational divergence as similarity measure is compared with the same method using classical metrics. In the astrophysics application, we also compare the method with the spectral clustering algorithms.

Motivation & Objective

• To address the challenge of initializing K-means clustering without prior knowledge of the number of clusters or their locations.
• To estimate the number of clusters in multi-dimensional data using the topological structure of a minimal spanning tree (MST).
• To develop a thresholding procedure on Prim’s trajectory to detect cluster modes based on statistical modeling of vertex distribution.
• To evaluate the performance of information-theoretic similarity measures (e.g., Kullback-Leibler divergence) in clustering multi-component data.
• To demonstrate the method’s effectiveness on real-world astrophysical and hyperspectral image datasets, outperforming spectral clustering and other baseline methods.

Proposed method

• Construct a minimal spanning tree (MST) using Prim’s algorithm on the data set to generate a one-dimensional trajectory of edge lengths.
• Model the distribution of vertices along the MST as a Poisson process under the null hypothesis to derive an analytical threshold for cluster detection.
• Identify cluster modes by detecting deep valleys in the Prim’s trajectory, corresponding to significant jumps in edge length.
• Use the detected cluster modes to estimate cluster centroids and initialize the K-means algorithm, avoiding random initialization bias.
• Employ symmetrical divergences, particularly Kullback-Leibler divergence, as similarity metrics in the feature space to enhance clustering accuracy.
• Apply the method to hyperspectral images and astrophysical data, using sub-sampling to reduce computational load while preserving cluster structure.

Experimental results

Research questions

• RQ1 Can the number of clusters in a multi-dimensional dataset be reliably estimated from the topological structure of a minimal spanning tree?
• RQ2 How can a statistically optimal threshold be derived for detecting cluster boundaries in the Prim’s trajectory of an MST?
• RQ3 To what extent do information-theoretic divergences improve clustering performance compared to classical metrics in hyperspectral and astrophysical data?
• RQ4 Can the proposed method eliminate the need for K-means initialization while maintaining or improving clustering accuracy?
• RQ5 How does the method perform relative to spectral clustering and other state-of-the-art clustering algorithms on real-world multi-spectral and astrophysical datasets?

Key findings

• The method successfully estimates the number of clusters by detecting valleys in the Prim’s trajectory, with a threshold derived from a Poisson model under the null hypothesis.
• Using Kullback-Leibler divergence as a similarity metric significantly improves clustering accuracy compared to classical metrics in both astrophysical and hyperspectral image applications.
• The proposed approach achieves results comparable to expert-driven methods like wavanglet and Bayesian positive source separation (BPSS), but without requiring reference spectra or prior expert labeling.
• On a Mars hyperspectral image, the method correctly identified three main components: H₂O ice, CO₂ ice, and dust, with spatial patterns consistent with physical expectations (e.g., H₂O ice at the periphery of CO₂ ice).
• The method reduces computational burden by avoiding full similarity matrix computation through data-driven hierarchical classification and sub-sampling of large images.
• The false positive rate of cluster detection is analytically bounded using the Poisson model, providing theoretical convergence rates and statistical confidence in cluster estimation.