[論文レビュー] DReS-FL: Dropout-Resilient Secure Federated Learning for Non-IID Clients via Secret Data Sharing

Jiawei Shao, Yuchang Sun+2 more|arXiv (Cornell University)|Oct 6, 2022

Privacy-Preserving Technologies in Data被引用数 21

ひとこと要約

DReS-FLは、ラグランジュ符号化計算と多項式整数ニューラルネットワークを用いたドロップアウト耐性のあるセキュアな連邦学習を導入し、非IIDでおそらくドロップするクライアントからでも、プライバシーを保ちながらバイアスのないグローバル勾配を可能にする。

ABSTRACT

Federated learning (FL) strives to enable collaborative training of machine learning models without centrally collecting clients' private data. Different from centralized training, the local datasets across clients in FL are non-independent and identically distributed (non-IID). In addition, the data-owning clients may drop out of the training process arbitrarily. These characteristics will significantly degrade the training performance. This paper proposes a Dropout-Resilient Secure Federated Learning (DReS-FL) framework based on Lagrange coded computing (LCC) to tackle both the non-IID and dropout problems. The key idea is to utilize Lagrange coding to secretly share the private datasets among clients so that each client receives an encoded version of the global dataset, and the local gradient computation over this dataset is unbiased. To correctly decode the gradient at the server, the gradient function has to be a polynomial in a finite field, and thus we construct polynomial integer neural networks (PINNs) to enable our framework. Theoretical analysis shows that DReS-FL is resilient to client dropouts and provides privacy protection for the local datasets. Furthermore, we experimentally demonstrate that DReS-FL consistently leads to significant performance gains over baseline methods.

研究の動機と目的

• Address non-IID data distribution across clients in federated learning.
• Mitigate the impact of arbitrary client dropouts on training performance.
• Provide privacy guarantees while enabling accurate global gradient decoding.
• Enable secure computation by making gradients polynomial over a finite field.

提案手法

• Encode private datasets with Lagrange polynomials to create secret shares distributed among clients.
• Use polynomial integer neural networks (PINNs) to ensure gradients are polynomials in a finite field.
• Perform local gradient computations on encoded mini-batches and decode the global gradient at the server via polynomial interpolation.
• Operate training in a finite field with quantized data and stochastic rounding to maintain integer parameters.
• Guarantee D-resilience (dropout tolerance), T-privacy (colluding privacy), and K-efficiency via (D,T,K) Lagrange coding constraints.
• Provide convergence analysis under L-smoothness and unbiased gradient assumptions.

実験結果

リサーチクエスチョン

• RQ1 Can DReS-FL ensure unbiased global gradients under arbitrary client dropouts while preserving privacy?
• RQ2 How does LCC-based secret data sharing interact with non-IID data to mitigate training instability?
• RQ3 What are the convergence properties of PINN-based federated learning in the DReS-FL framework?
• RQ4 What empirical gains does DReS-FL achieve compared to standard FedAvg, FedAvg-IS, and SCAFFOLD under non-IID and dropout conditions?

主な発見

データセット	FedAvg	FedAvg-IS	SCAFFOLD	DReS-FL（ours）	中央集権化
MNIST	96.17±0.05	97.06±0.10	71.89±3.92	97.38±0.08	97.99±0.04
Fashion-MNIST	81.20±0.07	85.94±0.16	55.22±1.83	86.60±0.32	89.02±0.11
EMNIST	71.50±0.28	77.09±0.34	55.15±5.95	78.04±0.29	82.45±0.23
CIFAR-10	89.54±0.09	89.83±0.07	54.17±9.13	90.31±0.19	90.37±0.12
CIFAR-100	67.71±0.26	68.92±0.14	29.97±1.73	69.15±0.27	71.12±0.09
SVHN	83.82±0.20	85.27±0.09	51.27±3.43	86.04±0.15	86.18±0.03

• DReS-FL consistently outperforms baseline methods on benchmark datasets under non-IID and dropout conditions.
• The approach provides strong privacy guarantees equivalent to secure aggregation while addressing data heterogeneity.
• PINNs enable the necessary polynomial gradient structure for Lagrange coding and secure decoding.
• Experiments show DReS-FL approaching centralized training performance in several settings.
• The framework supports dropout resilience with a tunable tradeoff among resiliency, privacy, and efficiency.