[Paper Review] Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints

Rishabh Iyer, Jeff Bilmes|arXiv (Cornell University)|Nov 8, 2013

Complexity and Algorithms in GraphsComputer Science40 references167 citations

TL;DR

This paper proposes a novel framework for solving two constrained submodular optimization problems: minimizing a submodular function subject to a submodular cover constraint (SCSC) and maximizing a submodular function subject to a submodular knapsack constraint (SCSK). It introduces scalable iterative algorithms (e.g., Gr, ISSC, ISK) and more precise approximation algorithms (e.g., EASSC, EASK), achieving bounded approximation guarantees and demonstrating strong empirical performance on real-world datasets, including sensor placement and data subset selection applications.

ABSTRACT

We investigate two new optimization problems -- minimizing a submodular function subject to a submodular lower bound constraint (submodular cover) and maximizing a submodular function subject to a submodular upper bound constraint (submodular knapsack). We are motivated by a number of real-world applications in machine learning including sensor placement and data subset selection, which require maximizing a certain submodular function (like coverage or diversity) while simultaneously minimizing another (like cooperative cost). These problems are often posed as minimizing the difference between submodular functions [14, 35] which is in the worst case inapproximable. We show, however, that by phrasing these problems as constrained optimization, which is more natural for many applications, we achieve a number of bounded approximation guarantees. We also show that both these problems are closely related and an approximation algorithm solving one can be used to obtain an approximation guarantee for the other. We provide hardness results for both problems thus showing that our approximation factors are tight up to log-factors. Finally, we empirically demonstrate the performance and good scalability properties of our algorithms.

Motivation & Objective

• Address the challenge of optimizing submodular functions under hard constraints, such as minimizing cost while ensuring coverage or maximizing utility under a budget.
• Overcome the limitations of prior approaches that frame such problems as minimizing the difference of submodular functions (DS optimization), which is inapproximable in the worst case.
• Provide a unified, constraint-based framework that enables bounded approximation guarantees for both submodular cover and submodular knapsack problems.
• Demonstrate the practical scalability and effectiveness of iterative algorithms (e.g., Gr, ISSC, ISK) compared to more complex but theoretically tighter algorithms (e.g., EASSC, EASK).
• Establish theoretical hardness results showing that the proposed approximation factors are tight up to logarithmic factors.

Proposed method

• Formulate two new optimization problems: SCSC (minimize f(X) subject to g(X) ≥ c) and SCSK (maximize g(X) subject to f(X) ≤ b), where f and g are monotone non-decreasing submodular functions.
• Introduce iterative greedy algorithms (Gr, ISSC, ISK) that build solutions incrementally by selecting elements with the highest marginal gain under the respective constraints.
• Propose more sophisticated approximation algorithms (EASSC, EASK) based on ellipsoidal approximations and surrogate functions to achieve tighter theoretical bounds.
• Leverage the duality between SCSC and SCSK, showing that an approximation algorithm for one can be adapted to provide guarantees for the other.
• Use curvature analysis and surrogate function construction to derive approximation ratios, with theoretical guarantees proven under monotonicity and normalization assumptions.
• Implement and evaluate algorithms on real-world instances (e.g., sensor placement, data subset selection), comparing performance against random selection and baseline methods.

Experimental results

Research questions

• RQ1 Can we achieve bounded approximation guarantees for submodular optimization problems with submodular cover and submodular knapsack constraints, despite the inapproximability of the standard difference-of-submodular (DS) formulation?
• RQ2 How do iterative greedy algorithms (e.g., Gr, ISSC, ISK) compare in practice to more complex approximation algorithms (e.g., EASSC, EASK) in terms of solution quality and runtime?
• RQ3 To what extent do the proposed algorithms scale to large-scale instances, particularly when compared to exact or branch-and-bound methods?
• RQ4 Can the duality between SCSC and SCSK be exploited to derive approximation guarantees for one problem using algorithms designed for the other?
• RQ5 What is the empirical performance of the proposed algorithms on real-world machine learning applications such as sensor placement and data subset selection?

Key findings

• The proposed iterative algorithms (Gr, ISSC, ISK) achieve solution quality comparable to the more complex EASSC and EASK algorithms in practice, despite having looser theoretical bounds.
• On real-world instances with |V| = 50, the iterative algorithms run in under a second, while the ellipsoidal approximation (EASSC) takes about 5 hours, highlighting a significant scalability advantage.
• All proposed algorithms significantly outperform random selection baselines in both solution quality and constraint satisfaction across tested datasets.
• Theoretical analysis shows that the approximation factors for both SCSC and SCSK are tight up to logarithmic factors, confirming the optimality of the bounds under standard assumptions.
• The framework successfully subsumes and generalizes prior problems such as submodular span and submodular knapsack with modular constraints, while providing stronger theoretical guarantees.
• Empirical results confirm that the algorithms are effective for real-world applications including sensor placement and limited vocabulary data subset selection in NLP and speech recognition.