[Paper Review] Submodular Maximization Beyond Non-negativity: Guarantees, Fast Algorithms, and Applications

Christopher Harshaw, Moran Feldman+2 more|arXiv (Cornell University)|Apr 19, 2019

Complexity and Algorithms in GraphsComputer Science32 references46 citations

TL;DR

The paper develops fast algorithms for maximizing g(S) - c(S) under cardinality (and unconstrained) settings when g is non-negative monotone and γ-weakly submodular while c is non-negative modular; it achieves (1−e^{−γ}−ε) g(OPT) − c(OPT) guarantees with runtimes independent of k, plus a γ-unknown handling and a matching hardness result.

ABSTRACT

It is generally believed that submodular functions -- and the more general class of $γ$-weakly submodular functions -- may only be optimized under the non-negativity assumption $f(S) \geq 0$. In this paper, we show that once the function is expressed as the difference $f = g - c$, where $g$ is monotone, non-negative, and $γ$-weakly submodular and $c$ is non-negative modular, then strong approximation guarantees may be obtained. We present an algorithm for maximizing $g - c$ under a $k$-cardinality constraint which produces a random feasible set $S$ such that $\mathbb{E} \left[ g(S) - c(S) ight] \geq (1 - e^{-γ} - ε) g(OPT) - c(OPT)$, whose running time is $O (\frac{n}ε \log^2 \frac{1}ε)$, i.e., independent of $k$. We extend these results to the unconstrained setting by describing an algorithm with the same approximation guarantees and faster $O(\frac{n}ε \log\frac{1}ε)$ runtime. The main techniques underlying our algorithms are two-fold: the use of a surrogate objective which varies the relative importance between $g$ and $c$ throughout the algorithm, and a geometric sweep over possible $γ$ values. Our algorithmic guarantees are complemented by a hardness result showing that no polynomial-time algorithm which accesses $g$ through a value oracle can do better. We empirically demonstrate the success of our algorithms by applying them to experimental design on the Boston Housing dataset and directed vertex cover on the Email EU dataset.

Motivation & Objective

• Motivate and address optimization when the objective is not non-negative by expressing it as f = g − c with g non-negative monotone γ-weakly submodular and c non-negative modular.
• Provide fast algorithms with strong approximation guarantees for cardinality-constrained maximization of g − c.
• Extend results to the unconstrained setting and analyze runtime efficiency.
• Introduce a hardness result showing limits of polynomial-time value-oracle approaches.

Proposed method

• Introduce a distorted surrogate objective Φ that reweights g and c over the algorithm’s progression: Φ_i(T) = (1−γ/k)^{k−i} g(T) − c(T).
• Define Ψ_i(T, e) = max{0, (1−γ/k)^{k−(i+1)} g(e|T) − c_e}.
• Develop Distorted Greedy which at each iteration adds the element maximizing the distorted gain and only accepts it if the distorted gain is positive.
• Introduce Stochastic Distorted Greedy which samples a subset B_i of the ground set to reduce evaluations while preserving guarantees.
• Provide a γ-Sweep meta-algorithm to handle unknown γ by geometrically sweeping candidate γ values and selecting the best resulting solution.
• Prove a matching hardness result showing no polynomial-time value-oracle algorithm can beat the presented guarantees.

Experimental results

Research questions

• RQ1 Can we achieve nontrivial approximation guarantees for maximizing g(S) − c(S) when g is γ-weakly submodular and c is non-negative modular?
• RQ2 How can we design fast algorithms whose runtime is independent of k for cardinality-constrained maximization of g − c?
• RQ3 What can be achieved when γ is unknown, and how can we robustly estimate or adapt to γ?
• RQ4 Is there a hardness barrier for polynomial-time value-oracle approaches to this problem?
• RQ5 Do these methods perform well in practical tasks such as experimental design and graph problems?

Key findings

• Distorted Greedy achieves (1 − e^{−γ}) g(OPT) − c(OPT) with O(nk) evaluations under known γ.
• Stochastic Distorted Greedy maintains the same guarantee in expectation with O(n log(1/ε)) evaluations.
• Unconstrained Distorted Greedy attains the same approximation with O(n) evaluations.
• When γ is unknown, γ-Sweep yields near-γ guarantees with a multiplicative overhead O((1/δ) log(1/δ)) in the number of subroutine calls.
• A hardness result shows no polynomial-time value-oracle algorithm can surpass these guarantees for γ-weakly submodular functions.
• Experiments on Boston Housing (experimental design) and Email EU (directed vertex cover) demonstrate practical effectiveness.