[Paper Review] Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator

Shengwen Yang, Bing Ren+2 more|arXiv (Cornell University)|Nov 22, 2019

Privacy-Preserving Technologies in Data8 references95 citations

TL;DR

The paper presents a parallel distributed logistic regression framework for vertical federated learning that removes the need for a third-party coordinator, leveraging a parameter-server architecture to enable scalable training across multiple servers.

ABSTRACT

Federated Learning is a new distributed learning mechanism which allows model training on a large corpus of decentralized data owned by different data providers, without sharing or leakage of raw data. According to the characteristics of data dis-tribution, it could be usually classified into three categories: horizontal federated learning, vertical federated learning, and federated transfer learning. In this paper we present a solution for parallel dis-tributed logistic regression for vertical federated learning. As compared with existing works, the role of third-party coordinator is removed in our proposed solution. The system is built on the pa-rameter server architecture and aims to speed up the model training via utilizing a cluster of servers in case of large volume of training data. We also evaluate the performance of the parallel distributed model training and the experimental results show the great scalability of the system.

Motivation & Objective

• Motivate and address training on vertically distributed data without sharing raw data.
• Develop a parallel distributed logistic regression solution suitable for vertical FL.
• Eliminate dependence on a third-party coordinator while maintaining privacy guarantees.
• Demonstrate scalability of the proposed system on large training datasets.

Proposed method

• Adopt a parameter-server-based architecture to coordinate distributed optimization.
• Implement parallel distributed logistic regression tailored for vertical federated learning.
• Remove reliance on a trusted third-party coordinator in the FL workflow.
• Utilize server clusters to speed up training for large-scale data.

Experimental results

Research questions

• RQ1 Can vertical federated learning be performed without a third-party coordinator while preserving model performance and privacy?
• RQ2 How does a parameter-server architecture affect training speed and scalability in vertical FL?
• RQ3 What is the impact of distributing logistic regression training across multiple servers on convergence?
• RQ4 Does the proposed approach achieve scalable performance compared with existing coordinator-dependent methods.

Key findings

• Demonstrates the feasibility of parallel distributed logistic regression in vertical FL without a third-party coordinator.
• Shows scalability of the system when using a cluster of servers for large training data volumes.
• Validates performance improvements with the parallel distributed approach in the vertical FL setting.