[Paper Review] Near Optimal Algorithms for Hard Submodular Programs with Discounted Cooperative Costs

Rishabh Iyer, Jeffrey A. Bilmes|arXiv (Cornell University)|Feb 26, 2019

Complexity and Algorithms in Graphs3 citations

TL;DR

This paper proposes near-optimal algorithms for hard submodular optimization problems—such as constrained submodular minimization, submodular coverage, and ratio minimization—by restricting cost functions to low-rank sums of concave-over-modular functions. The approach achieves constant-factor approximation guarantees where general submodular functions yield only Ω(√n) hardness, enabling strong theoretical guarantees for practical machine learning applications like cooperative image matching and sensor placement with discounted costs.

ABSTRACT

In this paper, we investigate a class of submodular problems which in general are very hard. These include minimizing a submodular cost function under combinatorial constraints, which include cuts, matchings, paths, etc., optimizing a submodular function under submodular cover and submodular knapsack constraints, and minimizing a ratio of submodular functions. All these problems appear in several real world problems but have hardness factors of $Ω(\sqrt{n})$ for general submodular cost functions. We show how we can achieve constant approximation factors when we restrict the cost functions to low rank sums of concave over modular functions. A wide variety of machine learning applications are very naturally modeled via this subclass of submodular functions. Our work therefore provides a tighter connection between theory and practice by enabling theoretically satisfying guarantees for a rich class of expressible, natural, and useful submodular cost models. We empirically demonstrate the utility of our models on real world problems of cooperative image matching and sensor placement with cooperative costs.

Motivation & Objective

• To address the inherent hardness of constrained submodular minimization and bi-optimization problems under general submodular cost functions, which suffer from Ω(√n) worst-case approximation factors.
• To identify a tractable yet expressive subclass of submodular functions—low-rank sums of concave-over-modular functions—that naturally model real-world cooperative costs in machine learning.
• To bridge the gap between theory and practice by providing constant-factor approximation guarantees for this subclass, where general submodular functions fail to offer such bounds.
• To empirically validate the proposed algorithms on real-world problems, including cooperative image matching and sensor placement with cooperative cost structures.

Proposed method

• The paper models submodular cost functions as f(X) = Σᵢ₌₁ᵏ ψᵢ(wᵢ(X)), where ψᵢ are monotone concave and wᵢ are modular functions, with k being constant or O(log n), forming a low-rank sum.
• It introduces a novel greedy algorithm, PLA (Primal-Linear Approximation), tailored for minimizing submodular cost under submodular coverage constraints, leveraging the low-rank structure.
• The approach uses a linear approximation of the concave components in the cost function to enable efficient optimization while preserving theoretical guarantees.
• Theoretical analysis shows that under the low-rank concave-over-modular assumption, the approximation factor improves from Ω(√n) to a constant, even for problems like submodular ratio minimization.
• The method is evaluated via two real-world applications: cooperative image matching using a submodular matching cost and sensor placement with group-based cost discounts.
• The algorithms are compared against greedy baselines (e.g., SGA and cost-agnostic greedy), demonstrating superior performance in both coverage and cost efficiency.

Experimental results

Research questions

• RQ1 Can we achieve constant-factor approximation for hard submodular optimization problems when the cost function is restricted to low-rank sums of concave-over-modular functions?
• RQ2 How does the low-rank structure of submodular cost functions enable improved theoretical approximation guarantees compared to general submodular functions?
• RQ3 Can this framework be effectively applied to real-world machine learning problems with cooperative costs, such as image matching and sensor placement?
• RQ4 How do the proposed algorithms (PLA, SGA) compare to standard greedy baselines in terms of objective value and cost efficiency in practical settings?

Key findings

• The paper achieves constant-factor approximation guarantees for constrained submodular minimization, submodular coverage, and ratio minimization problems under the low-rank concave-over-modular cost assumption, overcoming the Ω(√n) hardness of general submodular functions.
• The proposed PLA algorithm outperforms both SGA and cost-agnostic greedy algorithms in sensor placement, achieving higher mutual information coverage while effectively utilizing group-based cost discounts.
• In cooperative image matching, the low-rank submodular cost model significantly reduces mismatched point pairs compared to standard linear assignment, demonstrating improved matching quality.
• The piecewise-linear approximation of the cost function in sensor placement allows PLA to achieve exact optimization, confirming its effectiveness in settings with structured, non-additive costs.
• Empirical results show that PLA and SGA consistently outperform cost-agnostic greedy methods, especially when cooperative cost structures are present.
• The theoretical framework provides a strong, practical connection between submodular optimization theory and real-world applications involving economies of scale and cooperative costs.