[Paper Review] Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network

Joon-Woo Lee, HyungChul Kang+9 more|arXiv (Cornell University)|Jun 14, 2021

Cryptography and Data Security4 citations

TL;DR

This paper presents the first implementation of a standard ResNet-20 convolutional neural network using the RNS-CKKS fully homomorphic encryption (FHE) scheme with bootstrapping and advanced approximation methods for non-arithmetic activation functions. It achieves 90.67% classification accuracy on CIFAR-10 with 98.67% agreement to the plaintext model, demonstrating high fidelity and feasibility for privacy-preserving deep learning with state-of-the-art FHE.

ABSTRACT

Fully homomorphic encryption (FHE) is one of the prospective tools for privacypreserving machine learning (PPML), and several PPML models have been proposed based on various FHE schemes and approaches. Although the FHE schemes are known as suitable tools to implement PPML models, previous PPML models on FHE encrypted data are limited to only simple and non-standard types of machine learning models. These non-standard machine learning models are not proven efficient and accurate with more practical and advanced datasets. Previous PPML schemes replace non-arithmetic activation functions with simple arithmetic functions instead of adopting approximation methods and do not use bootstrapping, which enables continuous homomorphic evaluations. Thus, they could not use standard activation functions and could not employ a large number of layers. The maximum classification accuracy of the existing PPML model with the FHE for the CIFAR-10 dataset was only 77% until now. In this work, we firstly implement the standard ResNet-20 model with the RNS-CKKS FHE with bootstrapping and verify the implemented model with the CIFAR-10 dataset and the plaintext model parameters. Instead of replacing the non-arithmetic functions with the simple arithmetic function, we use state-of-the-art approximation methods to evaluate these non-arithmetic functions, such as the ReLU, with sufficient precision [1]. Further, for the first time, we use the bootstrapping technique of the RNS-CKKS scheme in the proposed model, which enables us to evaluate a deep learning model on the encrypted data. We numerically verify that the proposed model with the CIFAR-10 dataset shows 98.67% identical results to the original ResNet-20 model with non-encrypted data. The classification accuracy of the proposed model is 90.67%, which is pretty close to that of the original ResNet-20 CNN model...

Motivation & Objective

• To enable privacy-preserving inference of standard deep neural networks using fully homomorphic encryption (FHE).
• To overcome limitations of prior FHE-based PPML models that used simplified activation functions and lacked bootstrapping.
• To implement a full ResNet-20 model on encrypted data using the RNS-CKKS FHE scheme with high precision and accuracy.
• To demonstrate that advanced FHE techniques can be applied to real-world, complex models without retraining.

Proposed method

• Employed the RNS-CKKS FHE scheme with bootstrapping to support deep, continuous homomorphic evaluation of neural networks.
• Applied state-of-the-art approximation methods for non-arithmetic activation functions like ReLU, ensuring high precision.
• Integrated bootstrapping to manage noise growth and enable deeper network inference.
• Used parameter tuning and fine-tuning to balance accuracy, security, and performance.
• Leveraged the packing capability of RNS-CKKS to process multiple data points in parallel.
• Validated the encrypted model against the original ResNet-20 using the CIFAR-10 dataset and plaintext weights.

Experimental results

Research questions

• RQ1 Can standard deep learning models like ResNet-20 be securely and accurately executed on encrypted data using modern FHE schemes?
• RQ2 How accurately can non-arithmetic activation functions like ReLU be approximated in FHE without sacrificing model performance?
• RQ3 What is the impact of bootstrapping on the feasibility and performance of deep neural network inference in FHE?
• RQ4 Can the RNS-CKKS FHE scheme support end-to-end inference of a complex model like ResNet-20 with high classification accuracy?
• RQ5 How does the performance of FHE-based inference compare to plaintext inference in terms of accuracy and runtime?

Key findings

• The proposed FHE-based ResNet-20 achieved 90.67% classification accuracy on CIFAR-10, closely matching the 89.33% accuracy of the original model.
• The model showed 98.67% agreement ratio with the plaintext model across 10,000 test images, indicating high fidelity.
• The implementation successfully used bootstrapping in the RNS-CKKS scheme for the first time in a deep learning context, enabling deeper network evaluation.
• The running time for one image inference was approximately 4 hours on a dual Intel Xeon Platinum 8280 system with 112 cores and 512 GB RAM.
• The highest classification accuracy among prior word-wise FHE-based PPML models for CIFAR-10 was previously only 77%, making this a significant improvement.
• The security level was 98-bit, which can be increased by adjusting parameters at the cost of longer runtime.