[Paper Review] Vertical Federated Learning: Concepts, Advances and Challenges

Yang Liu, Yan Kang+7 more|arXiv (Cornell University)|Nov 23, 2022

Privacy-Preserving Technologies in Data179 references20 citations

TL;DR

A comprehensive survey of Vertical Federated Learning (VFL) covering concepts, architectures, privacy-preserving protocols, efficiency, effectiveness, security, and future directions through the proposed VFLow framework.

ABSTRACT

Vertical Federated Learning (VFL) is a federated learning setting where multiple parties with different features about the same set of users jointly train machine learning models without exposing their raw data or model parameters. Motivated by the rapid growth in VFL research and real-world applications, we provide a comprehensive review of the concept and algorithms of VFL, as well as current advances and challenges in various aspects, including effectiveness, efficiency, and privacy. We provide an exhaustive categorization for VFL settings and privacy-preserving protocols and comprehensively analyze the privacy attacks and defense strategies for each protocol. In the end, we propose a unified framework, termed VFLow, which considers the VFL problem under communication, computation, privacy, as well as effectiveness and fairness constraints. Finally, we review the most recent advances in industrial applications, highlighting open challenges and future directions for VFL.

Motivation & Objective

• Provide a thorough overview of VFL concepts, formulations, and training procedures.
• Categorize VFL settings and privacy-preserving protocols with analysis of attacks and defenses.
• Propose a unified optimization framework (VFLow) incorporating communication, computation, privacy, and fairness.
• Review strategies for improving efficiency, effectiveness, privacy, and security in VFL.
• Discuss industrial applications, challenges, and future research directions.

Proposed method

• Define the VFL problem with feature-space data partitioning and active/passive party roles.
• Present a unified VFL taxonomy including splitVFL, aggVFL, and variants with/without an active feature owner.
• Detail a two-step VFL training protocol: private entity alignment and privacy-preserving training with intermediate results.
• Survey privacy-preserving techniques (HE, MPC, DP, TEE) and associated attacks/defenses per protocol.
• Introduce the VFLow framework extending the VFL definition to accommodate communication, computation, privacy, effectiveness, and fairness.
• Discuss efficiency techniques (multiple updates, asynchronous coordination, one-shot communication, compression, sample/feature selection) and effectiveness enhancements (self-supervised learning, semi-supervised learning, and knowledge distillation).

Experimental results

Research questions

• RQ1 What are the core architectural variants and training protocols in vertically partitioned data settings?
• RQ2 How can privacy, efficiency, and effectiveness be simultaneously addressed in VFL across practical deployments?
• RQ3 What taxonomy and unified framework best capture the design space and trade-offs in VFL?
• RQ4 What are the prominent challenges and future directions for industrial applications of VFL?
• RQ5 How do privacy attacks/defenses differ across VFL protocols and how can they be mitigated?

Key findings

• The paper provides an exhaustive categorization of VFL settings and privacy-preserving protocols.
• A unified framework (VFLow) is proposed to consider communication, computation, privacy, effectiveness, and fairness.
• Privacy attacks and defense strategies are analyzed across different VFL protocols (HE, MPC, DP, TEE, etc.).
• Multiple efficiency techniques (local updates, asynchronous coordination, one-shot communication, compression, and data/feature selection) are reviewed with their trade-offs.
• Effectiveness enhancements through self-supervised learning, semi-supervised learning, and knowledge distillation are surveyed for leveraging unlabeled or unaligned data.
• The survey discusses industrial applications, open challenges, and directions toward building a VFL ecosystem.