[Paper Review] Towards Federated Clustering: A Federated Fuzzy $c$-Means Algorithm (FFCM)

Morris Stallmann, Anna Wilbik|arXiv (Cornell University)|Jan 18, 2022

Human Mobility and Location-Based Analysis19 citations

TL;DR

This paper proposes a Federated Fuzzy c-Means (FFCM) algorithm for clustering data across decentralized clients while preserving data privacy. It introduces two aggregation methods—federated averaging and k-means averaging—and demonstrates that k-means averaging outperforms standard averaging in identifying accurate global cluster centers, especially under non-i.i.d. data conditions, though at higher computational cost.

ABSTRACT

Federated Learning (FL) is a setting where multiple parties with distributed data collaborate in training a joint Machine Learning (ML) model while keeping all data local at the parties. Federated clustering is an area of research within FL that is concerned with grouping together data that is globally similar while keeping all data local. We describe how this area of research can be of interest in itself, or how it helps addressing issues like non-independently-identically-distributed (i.i.d.) data in supervised FL frameworks. The focus of this work, however, is an extension of the federated fuzzy $c$-means algorithm to the FL setting (FFCM) as a contribution towards federated clustering. We propose two methods to calculate global cluster centers and evaluate their behaviour through challenging numerical experiments. We observe that one of the methods is able to identify good global clusters even in challenging scenarios, but also acknowledge that many challenges remain open.

Motivation & Objective

• To address the challenge of clustering globally similar data across decentralized clients while maintaining data privacy.
• To extend the fuzzy c-means clustering algorithm to the federated learning (FL) setting, enabling unsupervised learning without centralizing data.
• To evaluate and compare two aggregation strategies—federated averaging and k-means averaging—for computing global cluster centers in a federated environment.
• To assess the performance of the proposed FFCM framework on synthetic, non-i.i.d. datasets with varying dimensions and cluster overlap.
• To identify open challenges in federated clustering, including cluster number estimation and initialization, and to motivate further research in this emerging area.

Proposed method

• The FFCM protocol follows the standard FL pipeline: client selection, model broadcast, local computation, aggregation, and model update.
• Each client performs local fuzzy c-means clustering on their private data, producing local cluster centers and membership degrees.
• Two aggregation methods are proposed: (1) standard federated averaging (avg₁) and (2) k-means averaging (avg₂), where global centers are computed by applying k-means to the local centers.
• The k-means averaging method treats local cluster centers as data points and re-clusters them to derive more robust global centers.
• The algorithm is iteratively applied until convergence, with global model updates shared across clients in each round.
• Performance is evaluated using knowledge gap and outside sum of squared errors (SSE) metrics on synthetic G2 benchmark datasets with varying dimensionality and standard deviation.

Experimental results

Research questions

• RQ1 Can a federated fuzzy c-means algorithm effectively identify global cluster centers without centralizing data?
• RQ2 How do different aggregation strategies—federated averaging versus k-means averaging—affect the accuracy of global clustering in non-i.i.d. data settings?
• RQ3 To what extent does the performance of FFCM degrade with increasing data dimensionality and cluster overlap?
• RQ4 How does the federated FFCM compare to non-federated fuzzy c-means in terms of cluster center recovery and knowledge gap?
• RQ5 What are the key open challenges in federated clustering, such as cluster number estimation and initialization, that remain unresolved?

Key findings

• The k-means averaging method (avg₂) consistently outperforms standard federated averaging (avg₁) in identifying accurate global cluster centers, particularly in high-overlap and high-dimensional scenarios.
• The knowledge gap for avg₂ was 30.38 at the 75th percentile, compared to 30.33 for avg₁, indicating comparable or slightly better performance in detecting true cluster structure.
• Both federated and non-federated fuzzy c-means achieved similar cluster center convergence across all test sets, confirming the viability of the federated approach.
• Performance degradation was observed with increasing standard deviation (higher cluster overlap) and dimensionality, as shown by rising knowledge gap and decreasing SSE-based separation.
• The maximum knowledge gap was 99.80 for non-federated, 99.24 for avg₁, and 95.55 for avg₂, indicating that avg₂ reduces error in extreme cases.
• Despite strong empirical results, the method assumes the number of clusters is known, and no principled evaluation of the fuzziness parameter m was performed.