[Paper Review] Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective

Yilun Jin, Xiguang Wei+2 more|arXiv (Cornell University)|Feb 26, 2020

Privacy-Preserving Technologies in DataComputer Science59 references58 citations

TL;DR

The paper surveys the use of unlabeled data in federated learning, outlining motivations, potential topics, and key challenges to guide future weakly supervised FL research.

ABSTRACT

Federated Learning (FL) proposed in recent years has received significant attention from researchers in that it can bring separate data sources together and build machine learning models in a collaborative but private manner. Yet, in most applications of FL, such as keyboard prediction, labeling data requires virtually no additional efforts, which is not generally the case. In reality, acquiring large-scale labeled datasets can be extremely costly, which motivates research works that exploit unlabeled data to help build machine learning models. However, to the best of our knowledge, few existing works aim to utilize unlabeled data to enhance federated learning, which leaves a potentially promising research topic. In this paper, we identify the need to exploit unlabeled data in FL, and survey possible research fields that can contribute to the goal.

Motivation & Objective

• Identify why unlabeled data is valuable in federated learning (FL) and where it is most needed.
• Review related weakly supervised learning paradigms applicable to FL (transfer, semi-, self-, and active learning).
• Propose research directions, scenarios, and challenges for integrating unlabeled data into FL.
• Discuss benefits such as mitigated non-iid domain shift and improved robustness in FL settings.

Proposed method

• Classify FL settings and participant types to frame unlabeled data opportunities (HFL, VFL, FTL);
• Survey existing weakly supervised learning methods and how they map to FL contexts;
• Discuss motivations and advantages of leveraging unlabeled data in FL, including domain discrepancy mitigation and robustness;
• Outline potential topics (transfer, semi-, self-, active learning) and associated challenges for future research;
• Provide a prospective agenda for scenarios and applications in weakly supervised FL.

Experimental results

Research questions

• RQ1 How can unlabeled data be effectively utilized to improve federated learning under privacy constraints?
• RQ2 What are the most promising weakly supervised paradigms for FL (transfer, semi-, self-, active learning) and what challenges do they face?
• RQ3 In which FL scenarios (cross-device vs cross-silo) is unlabeled data most beneficial, and why?
• RQ4 What are the key research directions and open problems for leveraging unlabeled data in FL?

Key findings

• Unlabeled data can help address non-iid domain shift and improve distributional understanding in FL.
• Weakly supervised methods (transfer, semi-, self-, active learning) have potential but face privacy, domain discrepancy, and scalability challenges in FL.
• Cross-device and cross-silo FL contexts present distinct opportunities and hurdles for unlabeled-data exploitation.
• There is a need for realistic federated datasets and evaluation methodologies to properly assess FTL and related approaches.
• Robustness and security considerations (e.g., adversarial attacks and membership inference) motivate regularization via unlabeled data.
• The paper outlines concrete research topics and challenges to guide future work in weakly supervised FL.