[Paper Review] A Centroid Auto-Fused Hierarchical Fuzzy c-Means Clustering

Yunxia Lin, Songcan Chen|arXiv (Cornell University)|Apr 27, 2020

Advanced Clustering Algorithms Research33 references4 citations

TL;DR

This paper proposes Centroid Auto-Fused Hierarchical Fuzzy c-Means (CAF-HFCM), a novel clustering method that automatically determines the optimal number of clusters through an adaptive fusion process during optimization, eliminating the need for external validity indices. By applying an ℓ₂-norm penalty on initial centroids and solving the problem via alternating optimization with ADMM, CAF-HFCM achieves fast convergence, reduced sensitivity to initialization, and robustness to noise, outperforming FCM and state-of-the-art methods on synthetic and real datasets.

ABSTRACT

Like k-means and Gaussian Mixture Model (GMM), fuzzy c-means (FCM) with soft partition has also become a popular clustering algorithm and still is extensively studied. However, these algorithms and their variants still suffer from some difficulties such as determination of the optimal number of clusters which is a key factor for clustering quality. A common approach for overcoming this difficulty is to use the trial-and-validation strategy, i.e., traversing every integer from large number like $\sqrt{n}$ to 2 until finding the optimal number corresponding to the peak value of some cluster validity index. But it is scarcely possible to naturally construct an adaptively agglomerative hierarchical cluster structure as using the trial-and-validation strategy. Even possible, existing different validity indices also lead to different number of clusters. To effectively mitigate the problems while motivated by convex clustering, in this paper we present a Centroid Auto-Fused Hierarchical Fuzzy c-means method (CAF-HFCM) whose optimization procedure can automatically agglomerate to form a cluster hierarchy, more importantly, yielding an optimal number of clusters without resorting to any validity index. Although a recently-proposed robust-learning fuzzy c-means (RL-FCM) can also automatically obtain the best number of clusters without the help of any validity index, so-involved 3 hyper-parameters need to adjust expensively, conversely, our CAF-HFCM involves just 1 hyper-parameter which makes the corresponding adjustment is relatively easier and more operational. Further, as an additional benefit from our optimization objective, the CAF-HFCM effectively reduces the sensitivity to the initialization of clustering performance. Moreover, our proposed CAF-HFCM method is able to be straightforwardly extended to various variants of FCM.

Motivation & Objective

• To address the persistent challenge of determining the optimal number of clusters in fuzzy c-means clustering without relying on trial-and-error validation indices.
• To develop a hierarchical clustering method that naturally emerges from the optimization process, avoiding manual or heuristic merging steps.
• To reduce sensitivity to initialization, a known weakness in FCM-based algorithms.
• To improve robustness to noisy data points, which often degrade clustering performance in standard FCM.
• To design a method with minimal hyperparameter tuning, enhancing practical usability.

Proposed method

• The method initializes with a large number of data points as centroids, typically set to √n, to enable hierarchical structure formation.
• An ℓ₂-norm penalty is applied to the distance between initial centroids in the objective function to encourage automatic fusion of similar clusters during optimization.
• The optimization is solved via an alternating strategy: the membership matrix is updated analytically, while the centroid matrix is updated using the ADMM method.
• The algorithm progressively fuses centroids as the penalty parameter γ increases, forming a natural hierarchy of clusters.
• The process automatically stabilizes at an optimal cluster count without requiring external validation indices.
• The framework is extensible to variants of FCM, including kernelized FCM (KFCM).

Experimental results

Research questions

• RQ1 Can a fuzzy c-means clustering method automatically determine the optimal number of clusters without relying on external validity indices?
• RQ2 Can the optimization process of FCM naturally generate a hierarchical cluster structure through centroid fusion?
• RQ3 Does the proposed method reduce sensitivity to initialization compared to standard FCM?
• RQ4 How does the method perform on noisy datasets compared to baseline FCM with optimal cluster count?
• RQ5 Can the method be efficiently extended to other FCM variants, such as kernelized FCM?

Key findings

• CAF-HFCM achieved a Rand Index (RI) of 0.9963 ± 0 on noisy uniform data, significantly outperforming FCM’s 0.9887 ± 0.0091 with the same cluster count.
• The Adjusted Rand Index (ARI) for CAF-HFCM was 0.9736 ± 0, compared to FCM’s 0.9217 ± 0.0630, indicating superior clustering stability and accuracy.
• Normalized Mutual Information (NMI) values were 0.9804 ± 0 for CAF-HFCM versus 0.9608 ± 0.0306 for FCM, confirming better partition consistency.
• CAF-HFCM demonstrated consistent optimal clustering results across multiple runs, with zero variance in RI, ARI, and NMI, indicating minimal sensitivity to initialization.
• The algorithm converged in just 5 iterations on the Gaussian Mixture dataset even with c=19 initial centroids, showing fast convergence.
• Visual comparisons (Figures 15 and 16) confirmed that CAF-HFCM consistently achieved optimal partitions even with noisy data, while FCM with optimal c=13 still produced suboptimal results.