[论文解读] PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments

Fan Mo, Hamed Haddadi+4 more|arXiv (Cornell University)|Apr 29, 2021

Privacy-Preserving Technologies in Data参考文献 68被引用 40

一句话总结

PPFL 提出了一种隐私保护的联邦学习框架，在客户端和服务器端的 TEEs 中训练 DNN 层，采用贪婪的逐层训练以应对受限的 TEE 内存，同时在降低通信轮次和适度开销的情况下实现可比的模型效用。

ABSTRACT

We propose and implement a Privacy-preserving Federated Learning ($PPFL$) framework for mobile systems to limit privacy leakages in federated learning. Leveraging the widespread presence of Trusted Execution Environments (TEEs) in high-end and mobile devices, we utilize TEEs on clients for local training, and on servers for secure aggregation, so that model/gradient updates are hidden from adversaries. Challenged by the limited memory size of current TEEs, we leverage greedy layer-wise training to train each model's layer inside the trusted area until its convergence. The performance evaluation of our implementation shows that $PPFL$ can significantly improve privacy while incurring small system overheads at the client-side. In particular, $PPFL$ can successfully defend the trained model against data reconstruction, property inference, and membership inference attacks. Furthermore, it can achieve comparable model utility with fewer communication rounds (0.54$ imes$) and a similar amount of network traffic (1.002$ imes$) compared to the standard federated learning of a complete model. This is achieved while only introducing up to ~15% CPU time, ~18% memory usage, and ~21% energy consumption overhead in $PPFL$'s client-side.

研究动机与目标

• 动机：在联邦学习中存在隐私风险，以及需要对数据重构、属性推断和成员资格推断等进行更强保护。
• 提出 PPFL 一种实用框架，在 TEEs 内训练每一层 DNN 以在 FL 过程中保护所有层。
• 实现逐层训练和安全聚合，以在保持模型效用的同时克服有限的 TEE 内存。

提出的方法

• 在客户端 TEEs 内使用贪婪的逐层训练和安全聚合来训练每一层 DNN。
• 使用两条安全通道：REE-to-REE 进行普通数据交换，TEE-to-TEE 进行私有层更新。
• 在服务器 TEE 内对层更新进行安全聚合，并应用 FedAvg 形成新的全局层。
• 支持在一个区块内训练多层，以更好地利用 TEE 内存并减少轮次。
• 利用模型分区执行在前向和后向传递期间在 REEs 与 TEEs 之间传送中间激活值。

实验结果

研究问题

• RQ1 Can PPFL prevent data reconstruction, property inference, and membership inference attacks for all trained DNN layers in FL?
• RQ2 What is the trade-off between privacy guarantees, model utility, and system overhead when using greedy layer-wise training inside TEEs?
• RQ3 How does PPFL compare to standard end-to-end FL in terms of rounds, network traffic, and client-side resource usage?

主要发现

• PPFL can defend the trained model against data reconstruction, property inference, and membership inference attacks (attacks degraded to random guessing or 50% precision).
• PPFL incurs about 0.54x fewer communication rounds and 1.002x similar network traffic compared to standard FL for training a complete model, with comparable ML performance when training only the first few layers.
• PPFL introduces up to ~15% CPU time, ~18% memory usage, and ~21% energy consumption overhead on the client side.
• Training all DNN layers with PPFL takes roughly 3x more delay than end-to-end FL, but layer-wise training can achieve similar utility when focusing on the initial layers.