[Paper Review] On Euclidean $k$-Means Clustering with $\alpha$-Center Proximity

Amit Deshpande, Anand Louis+1 more|arXiv (Cornell University)|Apr 28, 2018

Facility Location and Emergency Management3 citations

TL;DR

This paper studies Euclidean $k$-means clustering under the constraint of $\alpha$-center proximity, where each point is strictly closer to its own cluster center than to any other by a factor $\alpha > 1$. It presents a fixed-parameter tractable algorithm running in time exponential in $k$ and $1/(\alpha-1)$, but linear in data size and dimension, and proves NP-hardness of approximating the problem within a $1+\varepsilon_0$ factor for any $\varepsilon_0 > 0$, even under $\alpha$-center proximity.

ABSTRACT

$k$-means clustering is NP-hard in the worst case but previous work has shown efficient algorithms assuming the optimal $k$-means clusters are \\emph{stable} under additive or multiplicative perturbation of data. This has two caveats. First, we do not know how to efficiently verify this property of optimal solutions that are NP-hard to compute in the first place. Second, the stability assumptions required for polynomial time $k$-means algorithms are often unreasonable when compared to the ground-truth clusters in real-world data. A consequence of multiplicative perturbation resilience is \\emph{center proximity}, that is, every point is closer to the center of its own cluster than the center of any other cluster, by some multiplicative factor $\\alpha > 1$. We study the problem of minimizing the Euclidean $k$-means objective only over clusterings that satisfy $\\alpha$-center proximity. We give a simple algorithm to find the optimal $\\alpha$-center-proximal $k$-means clustering in running time exponential in $k$ and $1/(\\alpha - 1)$ but linear in the number of points and the dimension. We define an analogous $\\alpha$-center proximity condition for outliers, and give similar algorithmic guarantees for $k$-means with outliers and $\\alpha$-center proximity. On the hardness side we show that for any $\\alpha' > 1$, there exists an $\\alpha \\leq \\alpha'$, $(\\alpha >1)$, and an $\\varepsilon_0 > 0$ such that minimizing the $k$-means objective over clusterings that satisfy $\\alpha$-center proximity is NP-hard to approximate within a multiplicative $(1+\\varepsilon_0)$ factor.

Motivation & Objective

• To address the gap between theoretical intractability and practical efficiency in $k$-means clustering by focusing on a realistic structural assumption: $\alpha$-center proximity.
• To design an efficient algorithm for minimizing the $k$-means objective under the $\alpha$-center proximity constraint, which is a natural stability condition observed in real-world data.
• To extend the framework to clustering with outliers by introducing an analogous $\alpha$-center proximity condition for outlier handling.
• To establish the inherent computational hardness of approximating $\alpha$-center proximal $k$-means within any constant factor better than $1+\varepsilon_0$, despite the structural constraint.

Proposed method

• Propose a new problem formulation: minimize the $k$-means objective only over clusterings that satisfy $\alpha$-center proximity, defined as $\text{dist}(x, \mu_j) > \alpha \cdot \text{dist}(x, \mu_i)$ for all $x \in C_i$, $i \neq j$, and $\alpha > 1$.
• Design a dynamic programming-based algorithm that runs in time $2^{O(k + 1/(α-1))} \cdot \text{poly}(n,d)$, exponential in $k$ and $1/(α-1)$, but linear in the number of points $n$ and dimension $d$.
• Introduce an analogous $\alpha$-center proximity condition for clustering with outliers, ensuring a multiplicative gap between inlier and outlier distances to their respective centers.
• Use a reduction from the vertex cover problem to prove that approximating the $\alpha$-center proximal $k$-means problem within a factor of $1+\varepsilon_0$ is NP-hard for some $\varepsilon_0 > 0$, even when $\alpha$ is bounded above by any $\alpha' > 1$.
• Construct a family of instances with $\Omega(2^{\tilde{\Omega}(k/(α^2-1))})$ distinct $\alpha$-center proximal clusterings, showing the exponential dependence on $k$ and $1/(α-1)$ is necessary.

Experimental results

Research questions

• RQ1 Can we design an efficient algorithm for $k$-means clustering under the $\alpha$-center proximity constraint, despite the NP-hardness of the general problem?
• RQ2 Is the $\alpha$-center proximity condition sufficient to enable fixed-parameter tractable algorithms for $k$-means, and what is the dependence on $k$ and $\alpha$?
• RQ3 Can the $\alpha$-center proximity condition be extended to handle clustering with outliers, and does it preserve algorithmic tractability?
• RQ4 Is the $\alpha$-center proximal $k$-means problem approximable within a constant factor better than $1+\varepsilon_0$ for any $\varepsilon_0 > 0$, even under the structural constraint?
• RQ5 What is the maximum number of distinct $\alpha$-center proximal clusterings possible for a given $k$ and $\alpha$?

Key findings

• The paper presents a fixed-parameter tractable algorithm for $\alpha$-center proximal $k$-means clustering with running time $2^{O(k + 1/(α-1))} \cdot \text{poly}(n,d)$, which is exponential in $k$ and $1/(α-1)$ but linear in $n$ and $d$.
• The paper proves that for any $\alpha' > 1$, there exists an $\alpha \leq \alpha'$ such that minimizing the $k$-means objective over $\alpha$-center proximal clusterings is NP-hard to approximate within a factor of $1+\varepsilon_0$ for some $\varepsilon_0 > 0$.
• The number of distinct $\alpha$-center proximal clusterings for a given $k$ and $\alpha$ is at least $2^{\tilde{\Omega}(k/(α^2-1))}$, showing that the exponential dependence on $k$ and $1/(α-1)$ is necessary in the worst case.
• An analogous $\alpha$-center proximity condition for outliers is defined, and the same algorithmic guarantees are extended to $k$-means with outliers under this condition.
• The hardness result holds even when the cluster sizes are bounded below by $\omega n/k$ for some $\omega > 0$, indicating that the constraint does not make the problem tractable in the approximation sense.