[論文レビュー] Vertical Federated Learning: Challenges, Methodologies and Experiments

Wei Kang, Jun Li+6 more|arXiv (Cornell University)|Feb 9, 2022

Privacy-Preserving Technologies in Data被引用数 52

ひとこと要約

この論文は general Vertical Federated Learning (VFL) フレームワークを提示し、Horizontal FL と対比し、核となる課題を特定し、解決策を提案し、実世界データセット（Adult and Avazu）で実験を通じてそれらを検証する。

ABSTRACT

Recently, federated learning (FL) has emerged as a promising distributed machine learning (ML) technology, owing to the advancing computational and sensing capacities of end-user devices, however with the increasing concerns on users' privacy. As a special architecture in FL, vertical FL (VFL) is capable of constructing a hyper ML model by embracing sub-models from different clients. These sub-models are trained locally by vertically partitioned data with distinct attributes. Therefore, the design of VFL is fundamentally different from that of conventional FL, raising new and unique research issues. In this paper, we aim to discuss key challenges in VFL with effective solutions, and conduct experiments on real-life datasets to shed light on these issues. Specifically, we first propose a general framework on VFL, and highlight the key differences between VFL and conventional FL. Then, we discuss research challenges rooted in VFL systems under four aspects, i.e., security and privacy risks, expensive computation and communication costs, possible structural damage caused by model splitting, and system heterogeneity. Afterwards, we develop solutions to addressing the aforementioned challenges, and conduct extensive experiments to showcase the effectiveness of our proposed solutions.

研究の動機と目的

• Propose a general VFL framework and clarify differences from HFL.
• Identify security/privacy, computation/communication, structural, and system-heterogeneity challenges in VFL.
• Develop and discuss solutions for these challenges.
• Demonstrate the effectiveness of proposed solutions through experiments on real datasets.

提案手法

• Define a general VFL workflow with seven steps (PSI, BM-FP, forward transmission, TM-FP, TM-BP, backward transmission, BM-BP).
• Compare VFL and HFL in data characteristics, exchanged messages, and model structures.
• Discuss privacy-preserving options including DP, Secure MPC, and Homomorphic Encryption and their trade-offs.
• Propose enhanced communication schemes: transmission compression, model pruning, and data sampling.
• Address asynchronous VFL with heterogeneity via intelligent allocation and history-based updates.
• Analyze splitting design impacts on communication, privacy, and model performance.

実験結果

リサーチクエスチョン

• RQ1 What are the unique challenges of vertical federated learning compared with horizontal FL?
• RQ2 How can privacy and security be preserved in VFL without prohibitive cost?
• RQ3 How can communication and computation be reduced in VFL while maintaining model performance?
• RQ4 How does the way models are split across participants affect privacy, efficiency, and accuracy?

主な発見

• A general VFL framework is proposed and key differences from HFL are identified.
• Privacy-preserving techniques (DP, SMC, HE) have trade-offs in utility, security, and efficiency in VFL.
• Compression, pruning, and data sampling can significantly reduce communication cost, with measurable impact on performance depending on settings.
• Splitting design affects both computation/communication costs and model performance, with deeper splits generally increasing cost and potentially reducing accuracy.
• Experiments on Adult and Avazu datasets demonstrate relationships between privacy level, compression, and AUC performance under different configurations.