[Paper Review] Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning

Briland Hitaj, Giuseppe Ateniese+1 more|arXiv (Cornell University)|Feb 24, 2017

Privacy-Preserving Technologies in Data67 references224 citations

TL;DR

The paper introduces a GAN-based active inference attack that enables an insider to reconstruct private training data from victims in collaborative (distributed/federated) deep learning, even when sharing is obfuscated or differentially private.

ABSTRACT

Deep Learning has recently become hugely popular in machine learning, providing significant improvements in classification accuracy in the presence of highly-structured and large databases. Researchers have also considered privacy implications of deep learning. Models are typically trained in a centralized manner with all the data being processed by the same training algorithm. If the data is a collection of users' private data, including habits, personal pictures, geographical positions, interests, and more, the centralized server will have access to sensitive information that could potentially be mishandled. To tackle this problem, collaborative deep learning models have recently been proposed where parties locally train their deep learning structures and only share a subset of the parameters in the attempt to keep their respective training sets private. Parameters can also be obfuscated via differential privacy (DP) to make information extraction even more challenging, as proposed by Shokri and Shmatikov at CCS'15. Unfortunately, we show that any privacy-preserving collaborative deep learning is susceptible to a powerful attack that we devise in this paper. In particular, we show that a distributed, federated, or decentralized deep learning approach is fundamentally broken and does not protect the training sets of honest participants. The attack we developed exploits the real-time nature of the learning process that allows the adversary to train a Generative Adversarial Network (GAN) that generates prototypical samples of the targeted training set that was meant to be private (the samples generated by the GAN are intended to come from the same distribution as the training data). Interestingly, we show that record-level DP applied to the shared parameters of the model, as suggested in previous work, is ineffective (i.e., record-level DP is not designed to address our attack).

Motivation & Objective

• Demonstrate that collaborative deep learning can leak training data to insider adversaries.
• Show that a GAN can generate prototypical samples from private training data during the learning process.
• Argue that record-level differential privacy is insufficient to prevent such attacks in collaborative setups.
• Highlight the risks of applying differential privacy in collaborative deep learning and motivate centralized learning as a privacy-preserving alternative.

Proposed method

• Develop an active adversary that impersonates a collaborator in a distributed learning protocol.
• Train a GAN that uses feedback from the victim’s evolving model to generate samples resembling the victim’s private data.
• Exploit the real-time learning dynamics to reconstruct prototypical training samples from the victim’s class distribution.
• Demonstrate that the attack remains effective even when local parameters are obfuscated with differential privacy.
• Argue the attack works in white-box access scenarios and under parameter sharing where only a fraction of gradients is exchanged.

Experimental results

Research questions

• RQ1 Can an insider attacker reconstruct private training data of other participants in collaborative deep learning using GANs?
• RQ2 Does differential privacy applied to shared parameters prevent such GAN-based information leakage in collaborative settings?
• RQ3 Is the attack effective against CNNs and other architectures where model inversion is weak?
• RQ4 Does secure aggregation in federated learning mitigate the risk, or can insider adversaries still exploit learning dynamics?

Key findings

• A GAN-based attack can generate samples indistinguishable in distribution from a victim’s private data without accessing the originals.
• The attack remains effective even when parameters are obfuscated via differential privacy at the record level.
• The adversary can influence the learning process to coerce victims into revealing more detailed information.
• The threat applies to federated or decentralized learning as well as centralized led to privacy risks, since insider participants can cause leakage.
• GANs provide stronger information leakage than traditional model inversion approaches, especially for CNNs.