[Paper Review] Scalable Spectral Clustering with Group Fairness Constraints

Ji Wang, Ding Lu+2 more|arXiv (Cornell University)|Oct 28, 2022

Complex Network Analysis Techniques4 citations

TL;DR

This paper proposes s-FairSC, a scalable spectral clustering algorithm that enforces group fairness by integrating nullspace projection and Hotelling’s deflation to replace expensive dense matrix operations with sparse matrix-vector products. The method achieves 12× speedup over FairSC on moderate-sized datasets while maintaining comparable fairness and scalability, with only marginal computational overhead compared to standard spectral clustering.

ABSTRACT

There are synergies of research interests and industrial efforts in modeling fairness and correcting algorithmic bias in machine learning. In this paper, we present a scalable algorithm for spectral clustering (SC) with group fairness constraints. Group fairness is also known as statistical parity where in each cluster, each protected group is represented with the same proportion as in the entirety. While FairSC algorithm (Kleindessner et al., 2019) is able to find the fairer clustering, it is compromised by high costs due to the kernels of computing nullspaces and the square roots of dense matrices explicitly. We present a new formulation of underlying spectral computation by incorporating nullspace projection and Hotelling's deflation such that the resulting algorithm, called s-FairSC, only involves the sparse matrix-vector products and is able to fully exploit the sparsity of the fair SC model. The experimental results on the modified stochastic block model demonstrate that s-FairSC is comparable with FairSC in recovering fair clustering. Meanwhile, it is sped up by a factor of 12 for moderate model sizes. s-FairSC is further demonstrated to be scalable in the sense that the computational costs of s-FairSC only increase marginally compared to the SC without fairness constraints.

Motivation & Objective

• To address the high computational cost of FairSC, which relies on dense matrix square roots and nullspace computations.
• To develop a scalable alternative to FairSC that maintains group fairness in spectral clustering.
• To exploit sparsity in the fair SC model to reduce computational complexity.
• To ensure the new algorithm scales efficiently with dataset size, approaching the performance of standard spectral clustering.

Proposed method

• Reformulate FairSC's spectral computation using nullspace projection to enforce fairness constraints.
• Apply Hotelling’s deflation to iteratively extract eigenvectors without explicit matrix factorization.
• Replace dense matrix operations with sparse matrix-vector products to leverage sparsity in the data.
• Integrate fairness constraints into the spectral clustering objective via a modified Laplacian matrix.
• Use iterative solvers (e.g., power method) that operate efficiently on sparse matrices.
• Ensure the algorithm maintains fairness by preserving group proportionality in each cluster.

Experimental results

Research questions

• RQ1 Can spectral clustering with group fairness be made computationally scalable without sacrificing fairness quality?
• RQ2 How does the computational cost of fair spectral clustering scale with dataset size compared to standard spectral clustering?
• RQ3 Can sparse matrix operations replace dense matrix operations in fair spectral clustering while preserving fairness?
• RQ4 What is the performance gain of the new method over FairSC in terms of speed and scalability?
• RQ5 Does the proposed method maintain fairness levels comparable to FairSC on synthetic and real-world networks?

Key findings

• s-FairSC achieves 12× speedup over FairSC on the modified stochastic block model for moderate dataset sizes.
• On the LastFMNet dataset, s-FairSC is 7× faster than FairSC when k ≥ 5, and runs as fast as standard SC.
• The average balance metric shows that s-FairSC produces clustering fairness levels nearly identical to FairSC, confirming fairness preservation.
• s-FairSC only incurs a marginal increase in computational cost compared to standard spectral clustering, demonstrating strong scalability.
• On random Laplacian models, s-FairSC maintains scalability across dataset sizes from 5,000 to 10,000 nodes with varying k and h.
• The algorithm effectively leverages sparsity, making it suitable for large-scale fair clustering applications.