[論文レビュー] FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentation

Xiaokang Zhang, Xuran Xiong+2 more|arXiv (Cornell University)|Mar 8, 2026

Advanced Neural Network Applications被引用数 0

ひとこと要約

tldr: FedEU introduces evidential uncertainty into federated fine-tuning for remote sensing image segmentation, using client-specific feature embeddings and Top-k uncertainty-guided weighting to achieve robust, personalized, and communication-efficient federation of vision foundation models.

ABSTRACT

Remote sensing image segmentation (RSIS) in federated environments has gained increasing attention because it enables collaborative model training across distributed datasets without sharing raw imagery or annotations. Federated RSIS combined with parameter-efficient fine-tuning (PEFT) can unleash the generalization power of pretrained foundation models for real-world applications, with minimal parameter aggregation and communication overhead. However, the dynamic adaptation of pretrained models to heterogeneous client data inevitably increases update uncertainty and compromises the reliability of collaborative optimization due to the lack of uncertainty estimation for each local model. To bridge this gap, we present FedEU, a federated optimization framework for fine-tuning RSIS models driven by evidential uncertainty. Specifically, personalized evidential uncertainty modeling is introduced to quantify epistemic variations of local models and identify high-risk areas under local data distributions. Furthermore, the client-specific feature embedding (CFE) is exploited to enhance channel-aware feature representation while preserving client-specific properties through personalized attention and an element-aware parameter update approach. These uncertainty estimates are uploaded to the server to enable adaptive global aggregation via a Top-k uncertainty-guided weighting (TUW) strategy, which mitigates the impact of distribution shifts and unreliable updates. Extensive experiments on three large-scale heterogeneous datasets demonstrate the superior performance of FedEU. More importantly, FedEU enables balanced model adaptation across diverse clients by explicitly reducing prediction uncertainty, resulting in more robust and reliable federated outcomes. The source codes will be available at https://github.com/zxk688/FedEU.

研究の動機と目的

• Motivate federated fine-tuning of vision foundation models for pixel-wise remote sensing segmentation without sharing data.
• Quantify and leverage epistemic uncertainty to guide local training and global aggregation.
• Introduce client-specific feature embeddings for personalized representation learning across heterogeneous clients.
• Develop a Top-k uncertainty-guided weighting strategy to robustly aggregate updates under data distribution shifts.
• Demonstrate the effectiveness of FedEU on large-scale heterogeneous RSIS datasets.

提案手法

• Encoder-decoder RSIS model with a frozen encoder from Segment Anything Model (SAM) and a trainable PEFT module for fine-tuning.
• Client-specific feature embeddings (CFE) that enable channel-aware, personalized feature representations through a gated channel attention mechanism.
• Evidential uncertainty (EU) head to estimate epistemic uncertainty via a Dirichlet-based output and an associated loss combining segmentation and evidential terms.
• Uncertainty-guided local training objective combining segmentation loss with an evidential uncertainty loss (Bayesian risk with KL regularization).
• Uncertainty-guided global aggregation using Top-k uncertainty (TUW) to assign client weights based on pixel-wise uncertainty, promoting robust aggregation.
• Adaptive update of the client-specific feature embeddings (CFE) through an element-wise, learned fusion with global updates to preserve personalization.

実験結果

リサーチクエスチョン

• RQ1 How can evidential uncertainty be integrated into federated fine-tuning to handle heterogeneity in RSIS tasks?
• RQ2 Can client-specific feature embeddings improve channel-wise representation and personalization under data distribution shifts?
• RQ3 Does a Top-k uncertainty-guided weighting strategy improve robustness and convergence of federated RSIS models compared to standard aggregation?
• RQ4 What are the performance benefits of FedEU on heterogeneous RSIS datasets with non-IID client data?
• RQ5 How does the combination of PEFT with uncertainty-aware federation affect communication efficiency and robustness?

主な発見

• FedEU achieves superior performance over several federated baselines on three large-scale RSIS datasets.
• The framework enables balanced model adaptation across diverse clients by explicitly reducing prediction uncertainty.
• CFE and adaptive, uncertainty-aware aggregation contribute to robustness against spatial and spectral heterogeneity.
• Top-k uncertainty-guided weighting mitigates the impact of unreliable updates and distribution shifts during global aggregation.
• The approach demonstrates improved segmentation accuracy (IoU and OA) across multiple clients and datasets without sharing raw data.