[Paper Review] A Stackelberg Game Perspective on the Conflict Between Machine Learning and Data Obfuscation

Jeffrey Pawlick, Quanyan Zhu|arXiv (Cornell University)|Aug 8, 2016

Privacy-Preserving Technologies in Data13 references3 citations

TL;DR

This paper models the strategic conflict between machine learners and data-obfuscating users as an N+1-player Stackelberg game, where the learner commits to a privacy level first, and users respond by adding noise to their data. The key finding is that learners can improve accuracy and utility by proactively perturbing data themselves, mitigating the negative impact of user-level obfuscation and making privacy protection incentive-compatible for the learner.

ABSTRACT

Data is the new oil; this refrain is repeated extensively in the age of internet tracking, machine learning, and data analytics. As data collection becomes more personal and pervasive, however, public pressure is mounting for privacy protection. In this atmosphere, developers have created applications to add noise to user attributes visible to tracking algorithms. This creates a strategic interaction between trackers and users when incentives to maintain privacy and improve accuracy are misaligned. In this paper, we conceptualize this conflict through an N+1-player, augmented Stackelberg game. First a machine learner declares a privacy protection level, and then users respond by choosing their own perturbation amounts. We use the general frameworks of differential privacy and empirical risk minimization to quantify the utility components due to privacy and accuracy, respectively. In equilibrium, each user perturbs her data independently, which leads to a high net loss in accuracy. To remedy this scenario, we show that the learner improves his utility by proactively perturbing the data himself. While other work in this area has studied privacy markets and mechanism design for truthful reporting of user information, we take a different viewpoint by considering both user and learner perturbation.

Motivation & Objective

• To model the strategic interaction between machine learners and privacy-conscious users who obfuscate data as a game-theoretic problem.
• To analyze the equilibrium behavior when users respond to a learner's announced privacy level by choosing their own perturbation amounts.
• To quantify the trade-off between privacy (via differential privacy) and accuracy (via empirical risk minimization) in this strategic setting.
• To investigate whether and how learners can improve their utility by proactively perturbing data to counteract user obfuscation.
• To demonstrate that proactive learner perturbation can make privacy protection incentive-compatible and reduce overall accuracy loss.

Proposed method

• Models the interaction as an N+1-player augmented Stackelberg game, with the learner as the leader and N users as followers.
• Uses differential privacy (DP) to quantify privacy loss, parameterized by εp, and empirical risk minimization (ERM) to measure prediction accuracy.
• Introduces perturbation vectors vi (user) and wi (learner) added to input data, with the total noise ui = vi + wi.
• Derives bounds on the difference between unperturbed and perturbed classifiers using L2-regularized ERM and Lipschitz conditions on the loss function.
• Applies concentration inequalities and chi-squared tail bounds to probabilistically bound the expected loss difference due to noise.
• Establishes equilibrium conditions and analyzes the learner's optimal strategy under user response, showing that proactive perturbation can reduce net accuracy loss.

Experimental results

Research questions

• RQ1 How does user-level data obfuscation affect the accuracy of machine learning models in a strategic setting?
• RQ2 What is the equilibrium outcome when users independently choose perturbation levels in response to a learner's announced privacy level?
• RQ3 Can a machine learner improve their utility by proactively perturbing data, even when users are already obfuscating?
• RQ4 Under what conditions is privacy protection incentive-compatible for the learner when users are obfuscating?
• RQ5 How does the combination of user and learner perturbation affect the overall accuracy of the learning system?

Key findings

• In equilibrium, users independently perturb their data, leading to a high net loss in model accuracy due to the aggregation of independent noise.
• The learner's utility is reduced when users obfuscate, as the combined noise from both parties degrades model performance.
• Proactive perturbation by the learner can mitigate the negative impact of user obfuscation by reducing the effective noise level in the training data.
• When user obfuscation is costly, the learner can achieve better utility by perturbing data themselves, making privacy protection incentive-compatible.
• Theoretical bounds show that the expected loss difference between the optimal and perturbed classifier is proportional to the sum of squared noise magnitudes, with high probability.
• The analysis establishes that the learner can achieve a favorable equilibrium by strategically choosing perturbation levels that discourage excessive user obfuscation.