[论文解读] Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network

Joon-Woo Lee, HyungChul Kang+9 more|arXiv (Cornell University)|Jun 14, 2021

Cryptography and Data Security被引用 4

一句话总结

本文首次实现了使用RNS-CKKS全同态加密（FHE）方案结合自举技术与非算术激活函数的高级近似方法，对标准ResNet-20卷积神经网络进行同态加密实现。在CIFAR-10数据集上实现了90.67%的分类准确率，与明文模型的准确率一致率达到98.67%，证明了在最先进的FHE技术下，隐私保护深度学习具有高度保真度与可行性。

ABSTRACT

Fully homomorphic encryption (FHE) is one of the prospective tools for privacypreserving machine learning (PPML), and several PPML models have been proposed based on various FHE schemes and approaches. Although the FHE schemes are known as suitable tools to implement PPML models, previous PPML models on FHE encrypted data are limited to only simple and non-standard types of machine learning models. These non-standard machine learning models are not proven efficient and accurate with more practical and advanced datasets. Previous PPML schemes replace non-arithmetic activation functions with simple arithmetic functions instead of adopting approximation methods and do not use bootstrapping, which enables continuous homomorphic evaluations. Thus, they could not use standard activation functions and could not employ a large number of layers. The maximum classification accuracy of the existing PPML model with the FHE for the CIFAR-10 dataset was only 77% until now. In this work, we firstly implement the standard ResNet-20 model with the RNS-CKKS FHE with bootstrapping and verify the implemented model with the CIFAR-10 dataset and the plaintext model parameters. Instead of replacing the non-arithmetic functions with the simple arithmetic function, we use state-of-the-art approximation methods to evaluate these non-arithmetic functions, such as the ReLU, with sufficient precision [1]. Further, for the first time, we use the bootstrapping technique of the RNS-CKKS scheme in the proposed model, which enables us to evaluate a deep learning model on the encrypted data. We numerically verify that the proposed model with the CIFAR-10 dataset shows 98.67% identical results to the original ResNet-20 model with non-encrypted data. The classification accuracy of the proposed model is 90.67%, which is pretty close to that of the original ResNet-20 CNN model...

研究动机与目标

• 通过全同态加密（FHE）实现标准深度神经网络的安全私密推理。
• 克服先前基于FHE的PPML模型所存在的局限性，如使用简化的激活函数且缺乏自举机制。
• 在加密数据上使用RNS-CKKS FHE方案实现完整的ResNet-20模型，实现高精度与高准确率。
• 证明高级FHE技术可应用于无需微调的真实世界复杂模型。

提出的方法

• 采用RNS-CKKS FHE方案结合自举技术，支持神经网络的深度连续同态计算。
• 应用最先进的非算术激活函数（如ReLU）近似方法，确保高精度。
• 集成自举技术以管理噪声增长，支持更深网络的推理。
• 通过参数调优与微调，在准确率、安全性与性能之间实现平衡。
• 利用RNS-CKKS的打包能力并行处理多个数据点。
• 使用CIFAR-10数据集与明文权重，对加密模型与原始ResNet-20模型进行验证。

实验结果

研究问题

• RQ1 能否使用现代FHE方案在加密数据上安全且准确地执行标准深度学习模型（如ResNet-20）？
• RQ2 在不牺牲模型性能的前提下，非算术激活函数（如ReLU）在FHE中可实现多高的近似精度？
• RQ3 自举技术对FHE中深度神经网络推理的可行性与性能有何影响？
• RQ4 RNS-CKKS FHE方案能否支持复杂模型（如ResNet-20）的端到端推理并实现高分类准确率？
• RQ5 基于FHE的推理在准确率与运行时间方面与明文推理相比表现如何？

主要发现

• 所提出的基于FHE的ResNet-20在CIFAR-10上实现了90.67%的分类准确率，与原始模型的89.33%准确率非常接近。
• 在10,000张测试图像上，该模型与明文模型的预测结果一致率达到98.67%，表明其具有高度保真度。
• 该实现首次在深度学习场景中成功应用RNS-CKKS方案的自举技术，实现了更深网络的同态评估。
• 在配备112个核心与512 GB内存的双Intel Xeon Platinum 8280系统上，单张图像推理耗时约4小时。
• 此前基于FHE的PPML模型在CIFAR-10上的最高准确率为77%，本工作实现了显著提升。
• 安全级别为98位，可通过调整参数进一步提高，但会带来更长的运行时间。