[Paper Review] Differential Privacy and Machine Learning: a Survey and Review

Zhanglong Ji, Zachary C. Lipton+1 more|arXiv (Cornell University)|Dec 24, 2014

Privacy-Preserving Technologies in Data56 references193 citations

TL;DR

This paper surveys and reviews differentially private machine learning, proposing methods to balance privacy and utility in learning models. It presents techniques like noise addition, model modification, public data integration, and iterative noise for private learning, showing that privacy can be achieved at no utility cost under certain conditions, such as in regularized models with noise scaling as O(1/n).

ABSTRACT

The objective of machine learning is to extract useful information from data, while privacy is preserved by concealing information. Thus it seems hard to reconcile these competing interests. However, they frequently must be balanced when mining sensitive data. For example, medical research represents an important application where it is necessary both to extract useful information and protect patient privacy. One way to resolve the conflict is to extract general characteristics of whole populations without disclosing the private information of individuals. In this paper, we consider differential privacy, one of the most popular and powerful definitions of privacy. We explore the interplay between machine learning and differential privacy, namely privacy-preserving machine learning algorithms and learning-based data release mechanisms. We also describe some theoretical results that address what can be learned differentially privately and upper bounds of loss functions for differentially private algorithms. Finally, we present some open questions, including how to incorporate public data, how to deal with missing data in private datasets, and whether, as the number of observed samples grows arbitrarily large, differentially private machine learning algorithms can be achieved at no cost to utility as compared to corresponding non-differentially private algorithms.

Motivation & Objective

• To reconcile the conflicting goals of extracting useful information from data and protecting individual privacy in machine learning.
• To address the limitations of traditional anonymization and k-anonymity in preventing background attacks on sensitive datasets.
• To explore how differential privacy can be integrated into core machine learning tasks like classification, regression, clustering, and dimensionality reduction.
• To investigate whether differential privacy can be achieved at no cost to model utility as dataset size grows.
• To examine open challenges such as handling missing data, using public data, and designing private mechanisms for non-relational data like temporal or structural medical records.

Proposed method

• Uses differential privacy, defined as bounded sensitivity to individual data changes, to ensure that model outputs do not reveal individual information.
• Applies global sensitivity and Laplace mechanism to add noise to query outputs, ensuring privacy while preserving utility.
• Proposes noise addition to underlying counts (e.g., in naive Bayes) rather than high-sensitivity derived values to reduce noise magnitude.
• Introduces model transformation (e.g., kernel SVM to linear SVM) to reduce sensitivity and improve privacy-utility trade-offs.
• Leverages public datasets drawn from similar populations to reduce noise in private learning mechanisms, improving utility.
• Considers iterative noise addition in optimization paths, where per-iteration sensitivity is lower than final model sensitivity, enabling private training via MCMC or similar methods.

Experimental results

Research questions

• RQ1 Can differentially private machine learning algorithms be designed to maintain high utility, especially as dataset size increases?
• RQ2 Under what conditions can differential privacy be achieved at no cost to model utility, i.e., when privacy noise is asymptotically smaller than sample noise?
• RQ3 How can public data be effectively used to enhance the performance of differentially private learning mechanisms?
• RQ4 What are the challenges in applying differential privacy to incomplete, temporal, or structural datasets like medical records?
• RQ5 To what extent are the goals of differential privacy and model generalization compatible in machine learning?

Key findings

• For regularized logistic regression and linear SVM, differential privacy can be achieved with noise scaling as O(1/n), which is asymptotically smaller than the O(1/√n) noise from sample randomness, making privacy effectively free in utility terms.
• The mechanism in [6] achieves free ϵ-differential privacy for certain models, where privacy noise does not degrade performance relative to non-private baselines.
• Using public data can enhance the performance of differentially private mechanisms, as shown in [28, 29], by reducing the need for high noise in private data processing.
• Noise addition to low-sensitivity components (e.g., raw counts) rather than high-sensitivity derived values (e.g., conditional probabilities) significantly reduces noise magnitude and improves utility.
• Iterative noise addition in optimization paths can yield lower total sensitivity than final model sensitivity, enabling private training via MCMC-based algorithms.
• Differential privacy and model generalization are compatible: both aim to reduce overfitting to individual samples, suggesting a natural synergy between privacy and robustness in learning.