[Paper Review] A Survey towards Federated Semi-supervised Learning.

Yilun Jin, Xiguang Wei+2 more|arXiv (Cornell University)|Feb 26, 2020

Privacy-Preserving Technologies in Data15 references20 citations

TL;DR

This paper surveys federated semi-supervised learning (FSSL) as a promising direction to address the high cost of labeling in federated learning by leveraging unlabeled data across decentralized clients. It identifies gaps in current FL research, proposes integrating semi-supervised techniques into FL, and outlines key research areas that can enhance model performance with limited labeled data while preserving data privacy.

ABSTRACT

Federated Learning (FL) proposed in recent years has received significant attention from researchers in that it can bring separate data sources together and build machine learning models in a collaborative but private manner. Yet, in most applications of FL, such as keyboard prediction, labeling data requires virtually no additional efforts, which is not generally the case. In reality, acquiring large-scale labeled datasets can be extremely costly, which motivates research works that exploit unlabeled data to help build machine learning models. However, to the best of our knowledge, few existing works aim to utilize unlabeled data to enhance federated learning, which leaves a potentially promising research topic. In this paper, we identify the need to exploit unlabeled data in FL, and survey possible research fields that can contribute to the goal.

Motivation & Objective

• To address the high cost of labeling large-scale datasets in federated learning by exploring the use of unlabeled data.
• To identify the lack of existing research on semi-supervised learning in federated settings as a critical gap.
• To survey potential research fields and techniques that can enhance federated learning using unlabeled data.
• To propose a framework for integrating semi-supervised learning into federated learning for improved efficiency and privacy.

Proposed method

• Surveying existing federated learning frameworks and identifying limitations in handling unlabeled data.
• Analyzing semi-supervised learning techniques such as consistency regularization, pseudo-labeling, and mean teacher models for applicability in FL.
• Proposing a collaborative learning paradigm where clients use both labeled and unlabeled data to improve global model performance.
• Integrating self-training mechanisms in FL where models on clients generate pseudo-labels for unlabeled data to refine global updates.
• Adapting data augmentation and clustering strategies to improve generalization in low-labeled-data FL settings.
• Evaluating the feasibility of combining SSL with FL under communication and privacy constraints.

Experimental results

Research questions

• RQ1 How can unlabeled data be effectively utilized in federated learning to reduce reliance on costly labeling?
• RQ2 What semi-supervised learning techniques are most suitable for decentralized, privacy-preserving training in FL?
• RQ3 How can model generalization be improved in FL when only a small fraction of data is labeled?
• RQ4 What are the key challenges in integrating semi-supervised learning with federated learning architectures?
• RQ5 Which architectural and training modifications are necessary to ensure convergence and privacy in federated semi-supervised learning?

Key findings

• The paper identifies a significant research gap in leveraging unlabeled data within federated learning, despite its potential to reduce labeling costs.
• Semi-supervised learning techniques such as pseudo-labeling and consistency regularization are promising candidates for integration into FL frameworks.
• Federated semi-supervised learning can improve model accuracy with limited labeled data while maintaining data privacy.
• The integration of SSL into FL requires careful design to handle non-IID data and communication efficiency.
• Existing SSL methods need adaptation to work under the constraints of decentralized data and limited client updates in FL.
• The survey highlights that future research should focus on robust, communication-efficient, and privacy-preserving FSSL frameworks.