[Paper Review] Guarantees for Greedy Maximization of Non-submodular Functions with Applications

Yatao Bian, Joachim M. Buhmann+2 more|arXiv (Cornell University)|Mar 6, 2017

Complexity and Algorithms in Graphs31 references98 citations

TL;DR

The paper derives tight approximation guarantees for Greedy maximizing non-submodular nondecreasing set functions under a cardinality constraint, using a combination of submodularity ratio and curvature, and validates on several real-world objectives.

ABSTRACT

We investigate the performance of the standard Greedy algorithm for cardinality constrained maximization of non-submodular nondecreasing set functions. While there are strong theoretical guarantees on the performance of Greedy for maximizing submodular functions, there are few guarantees for non-submodular ones. However, Greedy enjoys strong empirical performance for many important non-submodular functions, e.g., the Bayesian A-optimality objective in experimental design. We prove theoretical guarantees supporting the empirical performance. Our guarantees are characterized by a combination of the (generalized) curvature $α$ and the submodularity ratio $γ$. In particular, we prove that Greedy enjoys a tight approximation guarantee of $\frac{1}α(1- e^{-γα})$ for cardinality constrained maximization. In addition, we bound the submodularity ratio and curvature for several important real-world objectives, including the Bayesian A-optimality objective, the determinantal function of a square submatrix and certain linear programs with combinatorial constraints. We experimentally validate our theoretical findings for both synthetic and real-world applications.

Motivation & Objective

• Motivate and analyze subset selection as maximizing a non-submodular, nondecreasing set function under a K-cardinality constraint.
• Introduce and leverage generalized curvature and submodularity ratio to bound Greedy performance.
• Provide tight approximation guarantees for Greedy in terms of curvature and submodularity ratio.
• Bound these parameters for important objectives like Bayesian A-optimality and determinantal functions.
• Demonstrate empirical validity on synthetic and real-world datasets.

Proposed method

• Define submodularity ratio (gamma) and generalized curvature (alpha) for non-submodular F.
• Prove the Greedy guarantee F(S^K) >= (1/alpha)[1 - ((K - alpha*gamma)/K)^K] F(Omega*) and the bound >= (1/alpha) (1 - e^{-alpha*gamma}) F(Omega*).
• Show that the worst-case guarantee is tight via LP-based arguments and groupings by overlap with Omega*.
• Provide interpretations recovering classical bounds when alpha or gamma take special values (e.g., submodular or supermodular cases).
• Bound gamma and alpha for Bayesian A-optimality, determinantal functions, and LPs with combinatorial constraints.
• Experimentally validate the theory on synthetic and real data, comparing Greedy to SDP-based methods and reporting runtime improvements.

Experimental results

Research questions

• RQ1 What are tight approximation guarantees for Greedy when maximizing non-submodular nondecreasing set functions under a K-cardinality constraint?
• RQ2 How can curvature and submodularity ratio be combined to characterize Greedy performance beyond submodular cases?
• RQ3 Can these parameters be bounded for important real-world objectives such as Bayesian A-optimality, determinantal objectives, and LPs with combinatorial constraints?
• RQ4 Do empirical results on synthetic and real datasets align with the theoretical guarantees and illustrate computational advantages?

Key findings

• Greedy achieves a tight approximation factor of (1/alpha)(1 - e^{-gamma alpha}) for cardinality constrained maximization of non-submodular nondecreasing functions.
• The performance guarantees recover classical bounds in special cases and show curvature can boost guarantees when gamma is high.
• For several objectives, including Bayesian A-optimality and determinantal functions, gamma and alpha can be bounded to yield practical guarantees for Greedy.
• Greedy often matches SDP-based methods in solution quality while offering orders of magnitude faster runtimes in experimental design settings.
• Empirical results on synthetic and real data corroborate that greedy submodularity ratio is close to 1 and greedy curvature is typically smaller than classical total curvature, aligning with observed performance.