[Paper Review] Deletion Robust Submodular Maximization over Matroids

Paul Dütting, Federico Fusco+3 more|arXiv (Cornell University)|Jan 31, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper presents the first constant-factor approximation algorithms for deletion-robust submodular maximization over general matroid constraints, achieving near-optimal space complexity. In the centralized setting, it offers a (3.582 + O(ε))-approximation with summary size O(k + d log k / ε²), while in the streaming setting, it achieves a (5.582 + O(ε))-approximation with the same space bounds, significantly improving upon prior Ω(kd) space requirements.

ABSTRACT

Maximizing a monotone submodular function is a fundamental task in machine learning. In this paper, we study the deletion robust version of the problem under the classic matroids constraint. Here the goal is to extract a small size summary of the dataset that contains a high value independent set even after an adversary deleted some elements. We present constant-factor approximation algorithms, whose space complexity depends on the rank $k$ of the matroid and the number $d$ of deleted elements. In the centralized setting we present a $(3.582+O(\varepsilon))$-approximation algorithm with summary size $O(k + \frac{d \log k}{\varepsilon^2})$. In the streaming setting we provide a $(5.582+O(\varepsilon))$-approximation algorithm with summary size and memory $O(k + \frac{d \log k}{\varepsilon^2})$. We complement our theoretical results with an in-depth experimental analysis showing the effectiveness of our algorithms on real-world datasets.

Motivation & Objective

• To design space-efficient algorithms for deletion-robust submodular maximization under general matroid constraints.
• To achieve constant-factor approximation guarantees despite adversarial deletion of up to d elements.
• To reduce the space complexity from prior Ω(kd) bounds to O(k + d log k / ε²), approaching the information-theoretic lower bound.
• To provide theoretical and empirical validation of the algorithms' performance across diverse real-world datasets.
• To close the gap between theoretical guarantees and practical performance by analyzing the impact of ε on robustness.

Proposed method

• The algorithm constructs a summary set W of size O(k + d log k / ε²) that includes elements likely to survive adversarial deletion.
• It uses a bucketing strategy based on element marginal gains, where elements are grouped by their contribution to the current solution.
• In the centralized setting, it applies a modified greedy selection with probabilistic sampling to ensure robustness against any d deletions.
• In the streaming setting, it maintains a dynamic summary using reservoir sampling and incremental updates to preserve approximation guarantees.
• The analysis leverages the (e/(e-1))-approximation factor (≈1.582) from the non-robust problem as a baseline, adding an additive cost of 2 or 4 for robustness.
• It introduces a novel analysis framework that accounts for worst-case deletion patterns while bounding the expected survival probability of selected elements.

Experimental results

Research questions

• RQ1 Can we achieve a constant-factor approximation for deletion-robust submodular maximization over general matroids with space complexity close to the information-theoretic minimum?
• RQ2 What is the minimal summary size required to maintain a constant-factor approximation under adversarial deletion of d elements?
• RQ3 How does the approximation guarantee degrade as the robustness parameter ε increases, and how does this compare to theoretical bounds?
• RQ4 Can we design streaming algorithms that maintain constant-factor approximation with space complexity independent of k and d in a multiplicative sense?
• RQ5 Why do empirical results significantly outperform theoretical worst-case bounds, and what does this imply about the adversary model's pessimism?

Key findings

• The centralized algorithm achieves a (3.582 + O(ε))-approximation with summary size O(k + d log k / ε²), improving upon prior O(kd) space complexity.
• The streaming algorithm achieves a (5.582 + O(ε))-approximation with the same space complexity, matching the theoretical lower bound up to logarithmic factors.
• Empirical results show that performance improves as ε decreases, with algorithms consistently outperforming theoretical worst-case bounds.
• In all experiments—on Facebook, MovieLens, RunInRome, and Uber datasets—the proposed algorithms achieve near-optimal performance, especially for small ε values.
• The performance gap between theory and practice is attributed to the pessimistic assumption that the adversary always deletes the most valuable elements, which is unlikely in practice.
• The algorithms demonstrate robustness and scalability, with stable performance across different data distributions and deletion patterns.