[Paper Review] Improving Robustness to Model Inversion Attacks via Mutual Information Regularization

Tianhao Wang, Yuheng Zhang+1 more|arXiv (Cornell University)|Sep 11, 2020

Adversarial Robustness in Machine Learning23 references6 citations

TL;DR

This paper proposes Mutual Information Regularization-based Defense (MID), a model-agnostic approach that reduces model inversion (MI) attack success by minimizing the mutual information between model inputs and predictions. MID achieves state-of-the-art utility-privacy tradeoff across linear models, decision trees, and neural networks in both blackbox and whitebox settings, outperforming differential privacy and model-specific defenses.

ABSTRACT

This paper studies defense mechanisms against model inversion (MI) attacks -- a type of privacy attacks aimed at inferring information about the training data distribution given the access to a target machine learning model. Existing defense mechanisms rely on model-specific heuristics or noise injection. While being able to mitigate attacks, existing methods significantly hinder model performance. There remains a question of how to design a defense mechanism that is applicable to a variety of models and achieves better utility-privacy tradeoff. In this paper, we propose the Mutual Information Regularization based Defense (MID) against MI attacks. The key idea is to limit the information about the model input contained in the prediction, thereby limiting the ability of an adversary to infer the private training attributes from the model prediction. Our defense principle is model-agnostic and we present tractable approximations to the regularizer for linear regression, decision trees, and neural networks, which have been successfully attacked by prior work if not attached with any defenses. We present a formal study of MI attacks by devising a rigorous game-based definition and quantifying the associated information leakage. Our theoretical analysis sheds light on the inefficacy of DP in defending against MI attacks, which has been empirically observed in several prior works. Our experiments demonstrate that MID leads to state-of-the-art performance for a variety of MI attacks, target models and datasets.

Motivation & Objective

• To address the lack of generalizable, high-utility defenses against model inversion (MI) attacks that protect all attributes simultaneously.
• To formalize the distributional privacy risk of MI attacks beyond just training data, recognizing threats to test data from the same distribution.
• To explain why differential privacy (DP) often fails to defend against MI attacks despite theoretical guarantees.
• To develop a tractable, model-agnostic regularization method that improves robustness without sacrificing model utility.

Proposed method

• Introduce a mutual information (MI) regularizer into the training loss to minimize the information about model inputs contained in predictions.
• Develop tractable approximations of the MI regularizer for linear regression, decision trees, and neural networks to enable practical deployment.
• Formulate a game-based definition of MI attacks to rigorously quantify information leakage and evaluate defense effectiveness.
• Use variational inference and neural estimators to approximate the intractable mutual information term in the loss function.
• Train models with the regularized loss to reduce the correlation between inputs and outputs, limiting adversary inference capabilities.
• Apply the defense to both whitebox and blackbox MI attack scenarios, evaluating robustness across multiple datasets and model architectures.

Experimental results

Research questions

• RQ1 Why does differential privacy fail to defend against model inversion attacks in practice, despite its theoretical privacy guarantees?
• RQ2 Can a model-agnostic defense mechanism be designed that improves robustness to MI attacks without degrading model utility?
• RQ3 How does mutual information between model inputs and predictions relate to the success of MI attacks?
• RQ4 To what extent can a regularizer based on mutual information improve the privacy-utility tradeoff across diverse models and datasets?
• RQ5 Does the privacy risk of MI attacks extend beyond training data to include test data from the same distribution?

Key findings

• MID significantly outperforms differential privacy (DP) in defending against both blackbox and whitebox MI attacks, especially when maintaining high model utility.
• For neural networks, MID reduces the quality of reconstructed images in whitebox attacks (e.g., GMI) to near-random levels, while DP still preserves recognizable facial features.
• In decision trees, MID achieves better robustness than the model-specific 'Priority' defense in most cases, except when model performance exceeds F1 > 0.57.
• MID reduces expected calibration error (ECE) more effectively than DP, indicating better confidence calibration under attack.
• Attack performance on training and test sets is nearly identical under MID and DP, suggesting MI attacks threaten the entire data distribution, not just training examples.
• Theoretical analysis confirms that DP's inefficacy against MI attacks stems from the fact that high noise levels required for privacy protection severely degrade model utility.