[Paper Review] Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions

Rishabh Iyer, Stefanie Jegelka+1 more|arXiv (Cornell University)|Nov 8, 2013

Complexity and Algorithms in Graphs52 references50 citations

TL;DR

This paper introduces curvature as a key structural parameter that determines the complexity of approximating, learning, and minimizing submodular functions. It provides curvature-dependent upper and lower bounds that significantly refine prior results, showing that algorithms achieve much better approximation factors for functions with low curvature, with empirical results closely matching theoretical predictions across multiple problem settings.

ABSTRACT

We investigate three related and important problems connected to machine learning: approximating a submodular function everywhere, learning a submodular function (in a PAC-like setting [53]), and constrained minimization of submodular functions. We show that the complexity of all three problems depends on the 'curvature' of the submodular function, and provide lower and upper bounds that refine and improve previous results [3, 16, 18, 52]. Our proof techniques are fairly generic. We either use a black-box transformation of the function (for approximation and learning), or a transformation of algorithms to use an appropriate surrogate function (for minimization). Curiously, curvature has been known to influence approximations for submodular maximization [7, 55], but its effect on minimization, approximation and learning has hitherto been open. We complete this picture, and also support our theoretical claims by empirical results.

Motivation & Objective

• To understand how curvature affects the complexity of approximating, learning, and minimizing submodular functions.
• To refine existing polynomial-time approximation bounds for submodular problems by incorporating curvature as a parameter.
• To close the theoretical gap in understanding curvature's role in minimization and learning, which had been previously studied only in maximization.
• To empirically validate that curvature strongly influences approximation performance in practice.
• To unify and extend prior results by showing curvature-dependent bounds are both necessary and sufficient for improved approximation factors.

Proposed method

• Introduce a curvature coefficient $\kappa_f$ that quantifies the deviation from modularity in submodular functions.
• Use black-box transformations of the function to derive curvature-dependent approximation and learning algorithms.
• Transform existing minimization algorithms to use surrogate functions tailored to curvature, improving approximation guarantees.
• Define a normalized submodular function $f^R_\kappa(X) = \kappa f(X) + (1-\kappa)|X|$ to control curvature while preserving structure.
• Prove lower bounds via reductions from known hard problems, such as perfect matching and minimum edge cover.
• Conduct empirical evaluations on synthetic submodular functions with tunable curvature to validate theoretical bounds.

Experimental results

Research questions

• RQ1 How does curvature influence the approximation factor for learning and minimizing submodular functions?
• RQ2 Can curvature-dependent bounds tighten the previously known polynomial-time approximation limits for submodular problems?
• RQ3 Is there a significant gap between theoretical bounds and empirical performance, especially for functions with low curvature?
• RQ4 Can curvature serve as a unifying parameter across submodular optimization problems, including minimization and learning?
• RQ5 How do existing algorithms like MUB and EA perform in practice as curvature varies?

Key findings

• The paper establishes a lower bound of $\Omega(n^{1/3})$ for PMAC learning and $\Omega(\sqrt{n}/\log n)$ for approximation, both refined by curvature.
• For the minimum submodular edge cover problem, the approximation factor is bounded below by $\frac{n^{1-3\epsilon}}{2+(n^{1-3\epsilon}-2)(1-\kappa_f)+2\delta\kappa_f}$, showing curvature dependence.
• Empirical results show that approximation factors closely follow theoretical bounds, improving significantly as curvature $\kappa$ decreases.
• For $\alpha \geq n^{2/3}$, the EA algorithm finds the optimal solution, demonstrating that high cardinality constraints and low curvature enable exact optimization.
• As $n$ grows, the theoretical and empirical bounds saturate at $1/(1-\kappa)$, indicating a constant approximation factor independent of $n$ for fixed $\kappa < 1$.
• The results confirm that curvature is a critical determinant of tractability: functions with low curvature are significantly easier to learn and optimize than those with $\kappa \approx 1$.