[Paper Review] Strong Coresets for k-Median and Subspace Approximation: Goodbye Dimension

Christian Sohler, David P. Woodruff|arXiv (Cornell University)|Sep 9, 2018

Sparse and Compressive Sensing Techniques17 references4 citations

TL;DR

This paper presents the first strong coresets for k-median and subspace approximation with a size of poly(k/ε), independent of both n and d, using a novel dimensionality reduction technique that generalizes prior work on p-norms for p ≥ 1. The method enables efficient coreset construction in O(nnz(A)) + (n+d)poly(k/ε) + exp(poly(k/ε)) time, achieving (1+ε)-approximation for all center sets simultaneously.

ABSTRACT

We obtain the first strong coresets for the $k$-median and subspace approximation problems with sum of distances objective function, on $n$ points in $d$ dimensions, with a number of weighted points that is independent of both $n$ and $d$; namely, our coresets have size $ ext{poly}(k/ε)$. A strong coreset $(1+ε)$-approximates the cost function for all possible sets of centers simultaneously. We also give efficient $ ext{nnz}(A) + (n+d) ext{poly}(k/ε) + \exp( ext{poly}(k/ε))$ time algorithms for computing these coresets. We obtain the result by introducing a new dimensionality reduction technique for coresets that significantly generalizes an earlier result of Feldman, Sohler and Schmidt \cite{FSS13} for squared Euclidean distances to sums of $p$-th powers of Euclidean distances for constant $p\ge1$.

Motivation & Objective

• To develop strong coresets for k-median and subspace approximation that are independent of data dimension d and sample size n.
• To overcome the limitation of prior coreset methods requiring size linear in d or n, especially for sum-of-distances objectives.
• To generalize Feldman et al.'s dimensionality reduction technique from squared ℓ2 distances to general ℓp distances for p ≥ 1.
• To achieve a (1+ε)-approximation for all possible center sets simultaneously, enabling robust use in downstream applications.
• To design an efficient coreset construction algorithm with near-linear time complexity in input sparsity and poly(k/ε) output size.

Proposed method

• Introduce a new dimensionality reduction framework that projects input points into a poly(k/ε)-dimensional subspace while preserving ℓp distances for p ≥ 1.
• Use randomized sketching and orthogonal projection to reduce the ambient dimension to O(k/ε²), preserving the cost of k-median and subspace approximation up to (1±ε).
• Apply a coreset construction (e.g., from [12] or [4]) in the reduced space to obtain a weighted set of size poly(k/ε) that approximates all center sets.
• Transform the coreset back to the original space via coordinate recovery, ensuring the final coreset preserves distances to any k-center set.
• Leverage the fact that orthogonal transformations preserve ℓ2 distances, allowing the coreset to be valid in the original space.
• Use sampling and rescaling matrices to efficiently select and weight the final coreset points from the projected data.

Experimental results

Research questions

• RQ1 Can strong coresets for k-median and subspace approximation be constructed with size independent of both n and d?
• RQ2 Can the dimensionality reduction technique used for ℓ2 distances be generalized to ℓp distances for p ≥ 1?
• RQ3 Is it possible to achieve a (1+ε)-approximation for all k-center sets simultaneously using a coreset of size poly(k/ε)?
• RQ4 What is the time complexity of constructing such a coreset, and can it be made nearly linear in input sparsity?
• RQ5 Can the coresets be computed efficiently in distributed or streaming settings using the proposed method?

Key findings

• The paper constructs the first strong coresets for k-median and subspace approximation with size poly(k/ε), independent of both n and d.
• The coreset construction runs in O(nnz(A)) + (n+d)poly(k/ε) + exp(poly(k/ε)) time, achieving a (1+ε)-approximation for all k-center sets simultaneously.
• The method generalizes prior work on ℓ2-based coresets to general ℓp distances for p ≥ 1, enabling broader applicability.
• For k-median, the coreset size is O(k² log k / ε⁴), and the construction ensures (1+ε)-relative error for all center sets.
• The approach enables efficient computation in low-stretch or sparse input settings, with near-linear time complexity in input sparsity.
• The correctness is achieved through a chain of approximations: original space → reduced subspace → coreset in reduced space → back to original space, each within ε error.