[Paper Review] Deletion-Robust Submodular Maximization at Scale

Ehsan Kazemi, Morteza Zadimoghaddam+1 more|arXiv (Cornell University)|Nov 20, 2017

Privacy-Preserving Technologies in Data25 references3 citations

TL;DR

This paper proposes the first memory-efficient, scalable algorithms for deletion-robust submodular maximization under cardinality constraints, achieving (1/2 − δ)-approximation in centralized and streaming settings, and 0.218 − δ in distributed settings, even when up to d adversarial elements are deleted. The methods use core-set construction and robust greedy selection to maintain performance under data deletion due to privacy or fairness constraints.

ABSTRACT

Can we efficiently extract useful information from a large user-generated dataset while protecting the privacy of the users and/or ensuring fairness in representation. We cast this problem as an instance of a deletion-robust submodular maximization where part of the data may be deleted due to privacy concerns or fairness criteria. We propose the first memory-efficient centralized, streaming, and distributed methods with constant-factor approximation guarantees against any number of adversarial deletions. We extensively evaluate the performance of our algorithms against prior state-of-the-art on real-world applications, including (i) Uber-pick up locations with location privacy constraints; (ii) feature selection with fairness constraints for income prediction and crime rate prediction; and (iii) robust to deletion summarization of census data, consisting of 2,458,285 feature vectors.

Motivation & Objective

• Address the lack of robustness in existing submodular optimization methods when data elements are deleted due to privacy or fairness concerns.
• Develop algorithms that maintain high utility even after adversarial deletion of up to d elements, without prior knowledge of which elements will be removed.
• Ensure constant-factor approximation guarantees (1/2 − δ) in centralized and streaming models, and 0.218 − δ in distributed models, independent of d.
• Minimize memory usage so that the solution size scales sublinearly with d, enabling large-scale deployment on massive datasets.
• Demonstrate practical utility in real-world applications involving privacy-preserving data summarization and fair feature selection.

Proposed method

• Propose Robust-Centralized: a centralized algorithm using a core-set construction to maintain (1/2 − δ)-approximation with memory O(k + d log k / δ²).
• Design Robust-Streaming: a streaming algorithm with O(k log k / δ + d log²k / δ³) memory, achieving (1/2 − δ)-approximation via robust greedy selection.
• Introduce Robust-Distributed: a distributed algorithm using m machines, achieving 0.218 − δ approximation with O(m(k + d log k / δ²)) memory.
• Propose Compact-Distributed: a memory-optimized variant achieving 0.109 − δ approximation with O(k + d log k / δ²) total memory.
• Use a similarity-based objective function for data summarization: f(S) = ∑_{x∈S} ∑_{x'∈S, x'≠x} (1 − ||x−x'||/√68), modeling pairwise similarity.
• Apply core-set techniques to precompute robust representative subsets that remain effective even after deletion of d elements.

Experimental results

Research questions

• RQ1 Can we design a scalable submodular maximization algorithm that remains effective after deletion of any d adversarial elements, without prior knowledge of which elements will be deleted?
• RQ2 How can we minimize memory usage in deletion-robust submodular optimization while maintaining constant-factor approximation guarantees?
• RQ3 Can we achieve robustness in streaming and distributed settings where data is too large to store in main memory?
• RQ4 How does the performance of deletion-robust algorithms compare to standard greedy methods when sensitive or biased features are removed?
• RQ5 To what extent can deletion-robust algorithms maintain utility in real-world applications involving privacy and fairness constraints?

Key findings

• Robust-Centralized and Robust-Streaming achieve (1/2 − δ)-approximation with memory growing only logarithmically in d, unlike prior methods that scale multiplicatively with d.
• On the Uber pickup locations dataset, the proposed algorithms outperform state-of-the-art methods under location privacy constraints, maintaining high utility after deletion of up to 10% of data.
• In the crime rate prediction task, the centralized and streaming algorithms achieved RMSE of 0.163 and 0.177, respectively, outperforming standard greedy (RMSE 0.193) even though they did not know which features would be deleted.
• On the Census1990 dataset (2.45M samples), Robust-Distributed achieved normalized objective values above 1.0, indicating it outperformed even the baseline that knew the deleted set in advance.
• The distributed algorithms stored only ≈4,500 items on average (450 per machine) to summarize 2.45M data points, demonstrating high memory efficiency under 80% deletion.
• Robust-Distributed maintained strong performance across all deletion strategies (random, gender-based), including when 80% of items were deleted, confirming robustness in extreme cases.