[Paper Review] Weakly Submodular Maximization Beyond Cardinality Constraints: Does Randomization Help Greedy?

Lin Chen, Moran Feldman+1 more|arXiv (Cornell University)|Jul 13, 2017

Complexity and Algorithms in Graphs18 citations

TL;DR

This paper introduces a randomized greedy algorithm, Residual Random Greedy, which achieves a $(1+1/\gamma)^{-2}$ approximation ratio for maximizing $\gamma$-weakly submodular functions under general matroid constraints—providing the first non-trivial theoretical guarantee for this class of problems beyond cardinality constraints. The method improves upon standard greedy in practice, as shown in applications like image explanation and video summarization.

ABSTRACT

Submodular functions are a broad class of set functions, which naturally arise in diverse areas. Many algorithms have been suggested for the maximization of these functions. Unfortunately, once the function deviates from submodularity, the known algorithms may perform arbitrarily poorly. Amending this issue, by obtaining approximation results for set functions generalizing submodular functions, has been the focus of recent works. One such class, known as weakly submodular functions, has received a lot of attention. A key result proved by Das and Kempe (2011) showed that the approximation ratio of the greedy algorithm for weakly submodular maximization subject to a cardinality constraint degrades smoothly with the distance from submodularity. However, no results have been obtained for maximization subject to constraints beyond cardinality. In particular, it is not known whether the greedy algorithm achieves any non-trivial approximation ratio for such constraints. In this paper, we prove that a randomized version of the greedy algorithm (previously used by Buchbinder et al. (2014) for a different problem) achieves an approximation ratio of $(1 + 1/\\gamma)^{-2}$ for the maximization of a weakly submodular function subject to a general matroid constraint, where $\\gamma$ is a parameter measuring the distance of the function from submodularity. Moreover, we also experimentally compare the performance of this version of the greedy algorithm on real world problems against natural benchmarks, and show that the algorithm we study performs well also in practice. To the best of our knowledge, this is the first algorithm with a non-trivial approximation guarantee for maximizing a weakly submodular function subject to a constraint other than the simple cardinality constraint. In particular, it is the first algorithm with such a guarantee for the important and broad class of matroid constraints.

Motivation & Objective

• Address the lack of theoretical guarantees for maximizing weakly submodular functions under constraints beyond cardinality.
• Extend the applicability of greedy-based algorithms to general matroid constraints, a broad and important class of constraints in combinatorial optimization.
• Demonstrate that randomization improves the theoretical and practical performance of greedy algorithms in weakly submodular settings.
• Provide a provable approximation guarantee for a widely used class of functions that generalize submodular functions, especially relevant in machine learning and data mining.
• Bridge the gap between theoretical performance and practical behavior of greedy algorithms in non-submodular settings.

Proposed method

• Adapt the Residual Random Greedy algorithm, originally designed for submodular maximization, to weakly submodular functions under matroid constraints.
• Use a randomized selection mechanism that prioritizes elements with high marginal gain relative to a residual function, improving robustness to non-submodularity.
• Introduce a novel analysis framework to bound the approximation ratio in terms of $\gamma$, the parameter measuring deviation from submodularity.
• Establish that the approximation ratio is $(1+1/\gamma)^{-2}$, which degrades gracefully as $\gamma$ increases (i.e., as the function becomes less submodular).
• Leverage matroid structure to ensure feasibility of the solution at each step while maintaining theoretical guarantees.
• Validate the method empirically on real-world problems such as gene splice site detection and video summarization, comparing against standard greedy and random selection.

Experimental results

Research questions

• RQ1 Can a randomized greedy algorithm achieve a non-trivial approximation ratio for maximizing $\gamma$-weakly submodular functions under general matroid constraints?
• RQ2 Does randomization improve the theoretical performance of greedy algorithms in weakly submodular maximization beyond cardinality constraints?
• RQ3 How does the Residual Random Greedy algorithm compare to standard greedy and random selection in practical applications involving weakly submodular objectives?
• RQ4 Is there a provable approximation guarantee for greedy-style algorithms in the weakly submodular setting with complex constraints like matroids?
• RQ5 Can the theoretical analysis be extended to non-monotone weakly submodular functions under matroid constraints?

Key findings

• The Residual Random Greedy algorithm achieves an approximation ratio of $(1+1/\gamma)^{-2}$ for maximizing $\gamma$-weakly submodular functions under general matroid constraints.
• This is the first non-trivial approximation guarantee for weakly submodular maximization under constraints other than cardinality constraints.
• In gene splice site detection and video summarization tasks, Residual Random Greedy outperforms both standard greedy and random selection in identifying relevant features.
• The algorithm successfully identifies the head region of a Bernese mountain dog in image explanation tasks, while standard greedy selects only the nose and includes irrelevant parts.
• The likelihood of correct label prediction increases steadily with the number of selected superpixels under Residual Random Greedy, outperforming standard greedy and random selection.
• Random selection achieves a maximum likelihood of only $4.39 \times 10^{-4}$, indicating negligible performance, while Residual Random Greedy maintains a significantly higher and growing likelihood.