[論文レビュー] Secure Byzantine-Robust Machine Learning

Lie He, Sai Praneeth Karimireddy+1 more|arXiv (Cornell University)|Jun 8, 2020

Privacy-Preserving Technologies in Data参考文献 45被引用数 39

ひとこと要約

安定な2サーバー集約フレームワークを提案し、分散学習における入力プライバシーとビザンチン耐性を達成し、フォールトトレランスとローカル差分プライバシーおよび既存の頑健ルールとの互換性を確保する。

ABSTRACT

Increasingly machine learning systems are being deployed to edge servers and devices (e.g. mobile phones) and trained in a collaborative manner. Such distributed/federated/decentralized training raises a number of concerns about the robustness, privacy, and security of the procedure. While extensive work has been done in tackling with robustness, privacy, or security individually, their combination has rarely been studied. In this paper, we propose a secure two-server protocol that offers both input privacy and Byzantine-robustness. In addition, this protocol is communication-efficient, fault-tolerant and enjoys local differential privacy.

研究の動機と目的

• 分散/フェデレーテッド学習において、プライバシーとロバスト性を同時に追求する。
• 正直だが好奇心旺盛なサーバーに対して入力プライバシーを保つ安全な集約プロトコルの開発。
• 既存のビザンチン耐性集約規則との整合性を保ちつつ、厳密性を維持。
• 実運用でフォールトトレランス、スケーラビリティ、低通信オーバーヘッドを保証。

提案手法

• 作業者は更新を2つの非協調サーバーと秘密分散共有する。
• Two-server 2PC-based secure aggregation computes either standard sum or a robust, distance-based aggregation using Beaver’s triples.
• For robustness, pairwise distances between updates are computed securely on the servers and fed to a robust aggregation oracle to select weights.
• The final aggregated update is reconstructed and applied to the public model, with exactness preserved relative to non-private robust methods.
• The protocol supports dropout/new joiners of workers and is designed to be communication-efficient (uplink within a factor of 2 of non-private).
• The framework can be combined with differential privacy, enabling locally differentially private mechanisms when integrated with DP-based training.

実験結果

リサーチクエスチョン

• RQ1 Can input privacy be achieved in a Byzantine-robust distributed learning setting without leaking individual updates?
• RQ2 How can distance-based robust aggregation rules be securely integrated into a two-server MPC framework?
• RQ3 Does the secure protocol preserve the exact results of non-private robust aggregation?
• RQ4 What are the communication and fault-tolerance properties of the proposed scheme in practice?
• RQ5 How does the approach interface with differential privacy in practice?

主な発見

• The proposed two-server secure aggregation yields the exact same result as the non-privacy-preserving robust aggregation (exactness).
• The protocol preserves strong input privacy, such that servers learn only the final aggregated update and, in robustness mode, pairwise distances between updates, not the updates themselves.
• The approach is fault-tolerant to worker dropouts and new joiners, with a one-round communication scheme and modest overhead.
• The communication overhead is within a factor of 2 of non-private methods, and server-to-server communication can be accelerated; robustness can be achieved with existing rules like Krum.
• The method is compatible with local differential privacy and can be integrated with DP-based training techniques.