[论文解读] Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective

Yilun Jin, Xiguang Wei+2 more|arXiv (Cornell University)|Feb 26, 2020

Privacy-Preserving Technologies in Data参考文献 59被引用 58

一句话总结

本文综述了在联邦学习中使用未标记数据的情况，概述动机、潜在主题以及关键挑战，以指导未来的弱监督FL研究。

ABSTRACT

Federated Learning (FL) proposed in recent years has received significant attention from researchers in that it can bring separate data sources together and build machine learning models in a collaborative but private manner. Yet, in most applications of FL, such as keyboard prediction, labeling data requires virtually no additional efforts, which is not generally the case. In reality, acquiring large-scale labeled datasets can be extremely costly, which motivates research works that exploit unlabeled data to help build machine learning models. However, to the best of our knowledge, few existing works aim to utilize unlabeled data to enhance federated learning, which leaves a potentially promising research topic. In this paper, we identify the need to exploit unlabeled data in FL, and survey possible research fields that can contribute to the goal.

研究动机与目标

• Identify why unlabeled data is valuable in federated learning (FL) and where it is most needed.
• Review related weakly supervised learning paradigms applicable to FL (transfer, semi-, self-, and active learning).
• Propose research directions, scenarios, and challenges for integrating unlabeled data into FL.
• Discuss benefits such as mitigated non-iid domain shift and improved robustness in FL settings.

提出的方法

• Classify FL settings and participant types to frame unlabeled data opportunities (HFL, VFL, FTL);
• Survey existing weakly supervised learning methods and how they map to FL contexts;
• Discuss motivations and advantages of leveraging unlabeled data in FL, including domain discrepancy mitigation and robustness;
• Outline potential topics (transfer, semi-, self-, active learning) and associated challenges for future research;
• Provide a prospective agenda for scenarios and applications in weakly supervised FL.

实验结果

研究问题

• RQ1 How can unlabeled data be effectively utilized to improve federated learning under privacy constraints?
• RQ2 What are the most promising weakly supervised paradigms for FL (transfer, semi-, self-, active learning) and what challenges do they face?
• RQ3 In which FL scenarios (cross-device vs cross-silo) is unlabeled data most beneficial, and why?
• RQ4 What are the key research directions and open problems for leveraging unlabeled data in FL?

主要发现

• Unlabeled data can help address non-iid domain shift and improve distributional understanding in FL.
• Weakly supervised methods (transfer, semi-, self-, active learning) have potential but face privacy, domain discrepancy, and scalability challenges in FL.
• Cross-device and cross-silo FL contexts present distinct opportunities and hurdles for unlabeled-data exploitation.
• There is a need for realistic federated datasets and evaluation methodologies to properly assess FTL and related approaches.
• Robustness and security considerations (e.g., adversarial attacks and membership inference) motivate regularization via unlabeled data.
• The paper outlines concrete research topics and challenges to guide future work in weakly supervised FL.