[Paper Review] Scalable Private Learning with PATE

Nicolas Papernot, Shuang Song+4 more|arXiv (Cornell University)|Feb 24, 2018

Privacy-Preserving Technologies in Data31 references247 citations

TL;DR

The paper scales Private Aggregation of Teacher Ensembles (PATE) to large output spaces using Gaussian-noise GNMax and selective aggregators (Confident-GNMax and Interactive-GNMax), achieving high utility with very strong privacy (ε < 1.0) on a large-scale Glyph task.

ABSTRACT

The rapid adoption of machine learning has increased concerns about the privacy implications of machine learning models trained on sensitive data, such as medical records or other personal information. To address those concerns, one promising approach is Private Aggregation of Teacher Ensembles, or PATE, which transfers to a "student" model the knowledge of an ensemble of "teacher" models, with intuitive privacy provided by training teachers on disjoint data and strong privacy guaranteed by noisy aggregation of teachers' answers. However, PATE has so far been evaluated only on simple classification tasks like MNIST, leaving unclear its utility when applied to larger-scale learning tasks and real-world datasets. In this work, we show how PATE can scale to learning tasks with large numbers of output classes and uncurated, imbalanced training data with errors. For this, we introduce new noisy aggregation mechanisms for teacher ensembles that are more selective and add less noise, and prove their tighter differential-privacy guarantees. Our new mechanisms build on two insights: the chance of teacher consensus is increased by using more concentrated noise and, lacking consensus, no answer need be given to a student. The consensus answers used are more likely to be correct, offer better intuitive privacy, and incur lower-differential privacy cost. Our evaluation shows our mechanisms improve on the original PATE on all measures, and scale to larger tasks with both high utility and very strong privacy ($\varepsilon$ &lt; 1.0).

Motivation & Objective

• Address privacy concerns in ML on sensitive data by scaling PATE to large-class and uncurated datasets.
• Develop and analyze new noisy aggregation mechanisms that improve privacy guarantees and utility.
• Provide data-dependent privacy accounting using Rényi Differential Privacy for Gaussian noise in GNMax.
• Demonstrate practicality on a large-scale glyph recognition task with imbalanced and mislabeled data.

Proposed method

• Introduce Gaussian NoisyMax (GNMax) aggregation by adding Gaussian noise to teacher vote counts and selecting the class with the highest noisy votes.
• Provide data-dependent Rényi differential privacy analysis for GNMax to obtain tighter per-query privacy guarantees.
• Develop Confident-GNMax, which privately checks for strong consensus before answering and may skip uncertain queries to save privacy budget.
• Develop Interactive-GNMax, which incorporates student predictions to decide whether to answer or reinforce the student’s prediction.
• Use a privacy-composition framework (Rényi DP) to bound the cumulative privacy loss across multiple queries and tasks.
• Extend evaluation to a large-scale, uncurated Glyph dataset with 150 output classes to assess privacy-utility tradeoffs.

Experimental results

Research questions

• RQ1 Can PATE scale to tasks with a large number of output classes and uncurated, imbalanced data while preserving strong privacy guarantees?
• RQ2 Do Gaussian noise-based GNMax and selective aggregators improve privacy costs and model utility compared to the original Laplace-based LNMax?
• RQ3 How effective is data-dependent privacy analysis (via Rényi DP) for GNMax in achieving tight privacy bounds?
• RQ4 Can the approach achieve high accuracy with ε ≈ 1 on realistic, large-scale classification tasks like Glyph?

Key findings

• Gaussian GNMax reduces the required noise footprint and tightens privacy costs per query compared to LNMax.
• Data-dependent RDP analysis yields per-query ε substantially below 1 for realistic δ, improving overall privacy cost.
• Confident-GNMax and Interactive-GNMax selectively answer queries, reducing the number of labeled queries and improving the privacy-utility tradeoff.
• Experiments on MNIST, SVHN, UCI Adult, and Glyph show improved privacy and accuracy, with Glyph demonstrating scalability to 150 output classes and robustness to mislabeling and imbalance.
• VAT-based semi-supervised learning on the Glyph task provides strong utility under the PATE framework.