[Paper Review] Oblivious Sampling Algorithms for Private Data Analysis

Sajin Sasy, Olga Ohrimenko|arXiv (Cornell University)|Sep 28, 2020

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes a private sampling-based query framework that leverages trusted execution environments (TEEs) and novel oblivious sampling algorithms to ensure strong differential privacy guarantees during data analysis. By designing memory-access-patterns-secure variants of sampling without replacement and Poisson sampling, the framework achieves comparable model accuracy to shuffling while incurring significantly lower privacy loss—demonstrated experimentally on MNIST and CIFAR-10 with up to 5.5× lower privacy budget (ε) for the same accuracy.

ABSTRACT

We study secure and privacy-preserving data analysis based on queries executed on samples from a dataset. Trusted execution environments (TEEs) can be used to protect the content of the data during query computation, while supporting differential-private (DP) queries in TEEs provides record privacy when query output is revealed. Support for sample-based queries is attractive due to \emph{privacy amplification} since not all dataset is used to answer a query but only a small subset. However, extracting data samples with TEEs while proving strong DP guarantees is not trivial as secrecy of sample indices has to be preserved. To this end, we design efficient secure variants of common sampling algorithms. Experimentally we show that accuracy of models trained with shuffling and sampling is the same for differentially private models for MNIST and CIFAR-10, while sampling provides stronger privacy guarantees than shuffling.

Motivation & Objective

• To address the challenge of preserving privacy in data analysis when using TEEs, where memory access patterns can leak sample indices.
• To design secure, data-oblivious sampling algorithms that hide the indices of sampled data records from observers, even in untrusted TEE environments.
• To evaluate whether sampling-based approaches offer stronger privacy guarantees than shuffling in differentially private machine learning.
• To provide a secure, end-to-end privacy-preserving query framework that protects data from both the TEE host and colluding data scientists.

Proposed method

• Designing a new oblivious sampling algorithm for sampling without replacement that ensures memory access patterns do not reveal sample indices.
• Developing a data-oblivious variant of Poisson sampling to preserve privacy during batch sampling in TEEs.
• Integrating the sampling algorithms into a TEE-based framework that supports differentially private queries on encrypted datasets.
• Using Intel SGX to enforce code and data integrity, with attestation to verify the correctness and confidentiality of the execution environment.
• Applying the sampling methods to train deep learning models on MNIST and CIFAR-10, comparing accuracy and privacy loss with shuffling-based training.
• Measuring total privacy loss (ε) using Rényi differential privacy and comparing across sampling methods and shuffling.

Experimental results

Research questions

• RQ1 Can oblivious sampling algorithms be designed to hide sample indices from memory access patterns in TEEs?
• RQ2 Does sampling without replacement or Poisson sampling provide stronger privacy guarantees than shuffling in differentially private machine learning?
• RQ3 How does the choice of sampling method affect model accuracy and privacy loss (ε) in private training of deep neural networks?
• RQ4 Can secure sampling be efficiently integrated into a TEE-based framework to support private data analysis with strong end-to-end privacy guarantees?

Key findings

• The proposed oblivious sampling algorithms successfully prevent leakage of sample indices through memory access patterns, even in untrusted TEE environments.
• For the same number of training epochs, sampling without replacement (SWO) incurred a total privacy loss of ε = 2.13 on MNIST and ε = 4.89 on CIFAR-10, significantly lower than shuffling (ε = 9.39).
• Poisson sampling achieved a total privacy loss of ε = 0.82 on MNIST, the lowest among all methods tested, while maintaining comparable model accuracy.
• Model accuracy for SWO and Poisson sampling was nearly identical to shuffling: 97.43% (SWO) vs. 97.5% (shuffling) on MNIST, and 79.0% (SWO) vs. 79.6% (shuffling) on CIFAR-10.
• The results confirm that sampling-based training provides stronger privacy amplification than shuffling, with up to 5.5× lower ε for the same accuracy on CIFAR-10.
• Smaller batch sizes further reduced privacy loss, confirming the benefit of sampling for minimizing ε in differentially private learning.