[Paper Review] Geometric Median in Nearly Linear Time

Michael B. Cohen, Yin Tat Lee+3 more|arXiv (Cornell University)|Jun 16, 2016

Computational Geometry and Mesh Generation3 references10 citations

TL;DR

This paper presents a novel nearly linear-time algorithm for computing a $(1+\epsilon)$-approximate geometric median in $\mathbb{R}^d$ using a customized long-step interior point method, achieving $O(nd\log^3 \frac{1}{\epsilon})$ time complexity. It also provides a faster $O(d\epsilon^{-2})$ algorithm via stochastic subgradient descent, both significantly improving upon prior superlinear-time methods.

ABSTRACT

In this paper we provide faster algorithms for solving the geometric median problem: given $n$ points in $\mathbb{R}^{d}$ compute a point that minimizes the sum of Euclidean distances to the points. This is one of the oldest non-trivial problems in computational geometry yet despite an abundance of research the previous fastest algorithms for computing a $(1+ε)$-approximate geometric median were $O(d\cdot n^{4/3}ε^{-8/3})$ by Chin et. al, $ ilde{O}(d\exp{ε^{-4}\logε^{-1}})$ by Badoiu et. al, $O(nd+\mathrm{poly}(d,ε^{-1})$ by Feldman and Langberg, and $O((nd)^{O(1)}\log\frac{1}ε)$ by Parrilo and Sturmfels and Xue and Ye. In this paper we show how to compute a $(1+ε)$-approximate geometric median in time $O(nd\log^{3}\frac{1}ε)$ and $O(dε^{-2})$. While our $O(dε^{-2})$ is a fairly straightforward application of stochastic subgradient descent, our $O(nd\log^{3}\frac{1}ε)$ time algorithm is a novel long step interior point method. To achieve this running time we start with a simple $O((nd)^{O(1)}\log\frac{1}ε)$ time interior point method and show how to improve it, ultimately building an algorithm that is quite non-standard from the perspective of interior point literature. Our result is one of very few cases we are aware of outperforming traditional interior point theory and the only we are aware of using interior point methods to obtain a nearly linear time algorithm for a canonical optimization problem that traditionally requires superlinear time. We hope our work leads to further improvements in this line of research.

Motivation & Objective

• To develop faster algorithms for computing the geometric median of $n$ points in $\mathbb{R}^d$.
• To overcome the limitations of prior methods that required superlinear time, especially $O((nd)^{O(1)}\log \frac{1}{\epsilon})$ or worse.
• To achieve nearly linear time complexity in both $n$ and $d$ for $(1+\epsilon)$-approximate geometric median computation.
• To extend the results to the weighted geometric median problem with non-negative weights.

Proposed method

• Design a novel long-step interior point method tailored to the geometric median problem, enabling nearly linear-time iterations.
• Implement each iteration of the interior point method in nearly linear time using efficient linear algebra and sampling techniques.
• Use stochastic subgradient descent with a sampling oracle to achieve $O(d\epsilon^{-2})$ time complexity.
• Apply a weight reduction technique to transform the weighted geometric median problem into an equivalent unweighted problem with $O(n\epsilon^{-1})$ points.
• Leverage the alias method for efficient weighted sampling during stochastic subgradient descent.
• Integrate robustness analysis to ensure convergence and accuracy under perturbations, enabling rigorous performance guarantees.

Experimental results

Research questions

• RQ1 Can the geometric median problem be solved in nearly linear time in $n$ and $d$ for $(1+\epsilon)$-approximate solutions?
• RQ2 Can interior point methods be adapted to achieve nearly linear time for a canonical optimization problem traditionally requiring superlinear time?
• RQ3 How can stochastic subgradient descent be rigorously analyzed and optimized for geometric median computation with provable convergence?
• RQ4 Can the weighted geometric median problem be reduced to the unweighted case with minimal overhead in time complexity?
• RQ5 What structural properties of the geometric median enable faster convergence via non-standard interior point variants?

Key findings

• The paper achieves an $O(nd\log^3 \frac{1}{\epsilon})$ time algorithm for computing a $(1+\epsilon)$-approximate geometric median, improving upon prior $O((nd)^{O(1)}\log \frac{1}{\epsilon})$ methods.
• An $O(d\epsilon^{-2})$ time algorithm is provided using stochastic subgradient descent, which is optimal up to logarithmic factors for the sampling-based approach.
• The authors present a non-standard long-step interior point method that outperforms traditional interior point theory, marking one of the few cases where interior point methods break the superlinear time barrier.
• The weighted geometric median problem is reduced to the unweighted case with $O(n\epsilon^{-1})$ points, enabling the same time bounds via implicit representation.
• The algorithm's runtime can be further optimized to $O(z\log^3 \frac{n}{\epsilon})$ where $z$ is the number of non-zero entries in the input points.
• The work establishes a new theoretical foundation for using interior point methods in nearly linear time for convex optimization, with potential for broader impact.