[Paper Review] PAC-Learning for Strategic Classification

Ravi Sundaram, Anil Vullikanti+2 more|arXiv (Cornell University)|Dec 6, 2020

Adversarial Robustness in Machine Learning38 references4 citations

TL;DR

This paper introduces a unified framework for strategic classification by proposing the strategic VC-dimension (SVC), which generalizes PAC-learnability to settings where data points have heterogeneous preferences over classification outcomes. It fully characterizes the statistical learnability and computational complexity of linear classifiers under strategic manipulation, showing that SVC is always bounded by the standard VC-dimension and proving NP-hardness of empirical risk minimization under general cost functions.

ABSTRACT

The study of strategic or adversarial manipulation of testing data to fool a classifier has attracted much recent attention. Most previous works have focused on two extreme situations where any testing data point either is completely adversarial or always equally prefers the positive label. In this paper, we generalize both of these through a unified framework for strategic classification, and introduce the notion of strategic VC-dimension (SVC) to capture the PAC-learnability in our general strategic setup. SVC provably generalizes the recent concept of adversarial VC-dimension (AVC) introduced by Cullina et al. arXiv:1806.01471. We instantiate our framework for the fundamental strategic linear classification problem. We fully characterize: (1) the statistical learnability of linear classifiers by pinning down its SVC; (2) its computational tractability by pinning down the complexity of the empirical risk minimization problem. Interestingly, the SVC of linear classifiers is always upper bounded by its standard VC-dimension. This characterization also strictly generalizes the AVC bound for linear classifiers in arXiv:1806.01471.

Motivation & Objective

• To address the gap in existing literature that treats only extreme cases—fully adversarial or uniformly positive preferences—by introducing a unified framework for strategic classification with arbitrary preference distributions.
• To formalize the concept of strategic VC-dimension (SVC) as a measure of statistical learnability in strategic settings where data points strategically alter features to influence classification outcomes.
• To characterize the statistical learnability of linear classifiers by deriving the exact value of their SVC, showing it is always upper bounded by the standard VC-dimension.
• To analyze the computational complexity of empirical risk minimization (ERM) in strategic classification, proving NP-hardness under general cost functions and preference sets.
• To extend prior work on adversarial VC-dimension (AVC) by showing that SVC generalizes AVC and provides tighter, more comprehensive bounds for linear classifiers.

Proposed method

• Proposes a novel notion of strategic VC-dimension (SVC) that captures the complexity of learning in strategic classification, where data points have real-valued preferences r ∈ ℝ over the positive label.
• Models strategic behavior as a Stackelberg game in which each data point rationally manipulates its features to maximize utility r·𝕀(h(z)=1) − c(z;x), where c is a cost function.
• Analyzes the statistical learnability of linear classifiers by deriving the SVC for the linear hypothesis class under general cost functions and preference sets.
• Proves that the SVC of linear classifiers is always upper bounded by their standard VC-dimension, generalizing prior results on adversarial VC-dimension (AVC).
• Reduces the partition problem to a strategic classification instance to prove NP-hardness of the empirical risk minimization (ERM) problem under general cost functions and preference sets.
• Constructs specific hard instances with labeled data points, cost functions, and preference values to demonstrate NP-hardness in both symmetric and asymmetric preference scenarios.

Experimental results

Research questions

• RQ1 How can we generalize existing strategic classification models that assume either fully adversarial or uniformly positive preferences into a unified framework?
• RQ2 What is the appropriate notion of VC-dimension that captures PAC-learnability in strategic classification with heterogeneous preferences?
• RQ3 How does the strategic VC-dimension (SVC) of linear classifiers relate to their standard VC-dimension?
• RQ4 Is the empirical risk minimization (ERM) problem for strategic linear classification computationally tractable, or is it NP-hard?
• RQ5 Can we construct a reduction from a known NP-hard problem (e.g., partition) to prove the hardness of ERM in strategic classification?

Key findings

• The strategic VC-dimension (SVC) of linear classifiers is always upper bounded by their standard VC-dimension, providing a strong statistical generalization guarantee.
• The proposed SVC generalizes the adversarial VC-dimension (AVC) introduced by Cullina et al. (2018), extending its applicability to non-adversarial and heterogeneous preference settings.
• The empirical risk minimization (ERM) problem for strategic linear classification is NP-hard, even when preferences are bounded and cost functions are quadratic.
• A reduction from the partition problem to a strategic classification instance proves NP-hardness of ERM, establishing computational intractability under general cost functions.
• The constructed hard instances demonstrate that strategic manipulation with mixed preferences and asymmetric cost functions can preserve NP-hardness, even when preferences are not fully adversarial.