[Paper Review] Federated Multi-Target Domain Adaptation

Chun-Han Yao, Boqing Gong+4 more|arXiv (Cornell University)|Aug 17, 2021

Domain Adaptation and Few-Shot Learning45 references4 citations

TL;DR

This paper proposes DualAdapt, a novel Federated Multi-Target Domain Adaptation (FMTDA) framework that enables efficient, privacy-preserving model training using unlabeled client data and a centralized labeled dataset. By decoupling local adaptation on clients (via lightweight classifiers and GMMs) from global feature adaptation on the server using a mixup-based proxy set, DualAdapt achieves high accuracy with minimal communication and computational costs—reducing client FLOPs by ~75% compared to baselines.

ABSTRACT

Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy. However, it is not always feasible to obtain high-quality supervisory signals from users, especially for vision tasks. Unlike typical federated settings with labeled client data, we consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server. We further take the server-client and inter-client domain shifts into account and pose a domain adaptation problem with one source (centralized server data) and multiple targets (distributed client data). Within this new Federated Multi-Target Domain Adaptation (FMTDA) task, we analyze the model performance of exiting domain adaptation methods and propose an effective DualAdapt method to address the new challenges. Extensive experimental results on image classification and semantic segmentation tasks demonstrate that our method achieves high accuracy, incurs minimal communication cost, and requires low computational resources on client devices.

Motivation & Objective

• To address the practical challenge of training accurate vision models in federated learning when client data is unlabeled and private, while a centralized labeled dataset is available.
• To identify key challenges in FMTDA: lack of client data access on the server, inter-client domain shifts, limited client data, and communication/computation constraints.
• To propose a new framework that decouples lightweight local adaptation on clients from heavy global adaptation on the server, avoiding direct data transfer.
• To minimize client-side computational and communication costs while maintaining high model accuracy across diverse client domains.
• To evaluate the method on image classification and semantic segmentation tasks under realistic federated settings with strong baselines.

Proposed method

• Decompose the model into a frozen feature extractor and a trainable local classifier on each client device to reduce client-side computation.
• Fit a parametric Gaussian Mixture Model (GMM) on each client’s unlabeled data to encode its statistical distribution for transfer to the server.
• Upload only the GMM parameters and local classifier weights to the server, avoiding raw data transmission and preserving privacy.
• On the server, use a mixup-augmented version of the source dataset as a proxy set to simulate diverse target domains.
• Weight the mixup proxy samples using the client-specific GMMs to approximate each target domain’s distribution during global feature adaptation.
• Jointly fine-tune the global feature extractor and classifier on the server using the GMM-weighted proxy set, enabling effective domain adaptation without accessing client data.

Experimental results

Research questions

• RQ1 How do existing centralized domain adaptation methods perform when adapted to a federated, multi-target setting with unlabeled client data and a centralized labeled source?
• RQ2 What are the key challenges introduced by inter-client domain shifts and limited client data in federated multi-target domain adaptation?
• RQ3 Can a decoupled client-server training strategy reduce communication and computational costs while maintaining high model accuracy in FMTDA?
• RQ4 How effective is using a GMM-weighted mixup proxy set to approximate diverse client data distributions on the server without direct data access?
• RQ5 What is the optimal trade-off between model performance, communication cost, and client-side computation in FMTDA?

Key findings

• DualAdapt achieves a 1.1% accuracy gain over Fed-MCD on the Digit-Five benchmark while reducing client-side FLOPs by approximately 75% and communication overhead by 50%.
• Adding self-training to DualAdapt improves accuracy by an additional 0.7%, with negligible increase in computational cost.
• The GMM weighting mechanism enables the model to achieve near-oracle performance with only 1,000 additional parameters for upload and broadcast.
• The method reduces client-side computational cost to approximately 1/4 of that of federated baselines by freezing the feature extractor and avoiding full model updates on-device.
• In centralized multi-target DA experiments, the one-to-multiple method (shared feature extractor with private classifiers) outperforms one-to-one and one-to-combined approaches under low-data regimes, validating the design choice in FMTDA.
• Extensive experiments on image classification and semantic segmentation show that DualAdapt consistently outperforms both centralized and federated baselines in accuracy, communication efficiency, and computational efficiency.