[Paper Review] Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles

Zijian Zhang, Shuai Wang+3 more|arXiv (Cornell University)|Mar 5, 2021

Indoor and Outdoor Localization TechnologiesEngineering30 references69 citations

TL;DR

This paper proposes a federated learning (FL)-based dynamic map fusion framework for intelligent networked vehicles that enables distributed, online learning with missing labels. By combining a three-stage fusion scheme (DBSCAN clustering, score-based weighting, IoU pruning), FL for distributed feature model fine-tuning, and knowledge distillation for label generation, the framework achieves high map quality with low communication overhead, outperforming existing methods in simulation even without ground-truth labels.

ABSTRACT

The technology of dynamic map fusion among networked vehicles has been developed to enlarge sensing ranges and improve sensing accuracies for individual vehicles. This paper proposes a federated learning (FL) based dynamic map fusion framework to achieve high map quality despite unknown numbers of objects in fields of view (FoVs), various sensing and model uncertainties, and missing data labels for online learning. The novelty of this work is threefold: (1) developing a three-stage fusion scheme to predict the number of objects effectively and to fuse multiple local maps with fidelity scores; (2) developing an FL algorithm which fine-tunes feature models (i.e., representation learning networks for feature extraction) distributively by aggregating model parameters; (3) developing a knowledge distillation method to generate FL training labels when data labels are unavailable. The proposed framework is implemented in the Car Learning to Act (CARLA) simulation platform. Extensive experimental results are provided to verify the superior performance and robustness of the developed map fusion and FL schemes.

Motivation & Objective

• To address sensing and model uncertainties in dynamic map fusion for intelligent networked vehicles.
• To enable online, distributed learning when real-time data labels are missing.
• To reduce communication overhead while maintaining high map quality through efficient fusion and model aggregation.
• To develop a scalable, robust framework for real-time dynamic map fusion using LiDAR data and edge-based coordination.

Proposed method

• A three-stage fusion pipeline: (1) DBSCAN for spatial clustering of object detections, (2) score-based weighted averaging of overlapping objects, and (3) IoU-based box pruning to resolve conflicts.
• A point-cloud-based federated learning algorithm that aggregates model parameters from vehicles to fine-tune shared feature extraction networks without sharing raw data.
• A knowledge distillation method that uses the edge server’s fused map as a teacher model to generate pseudo-labels for local training data.
• Integration of FL and knowledge distillation into an ensemble distillation federated learning (EDFL) framework to improve model robustness under label scarcity.
• Implementation and evaluation in the CARLA simulation platform using LiDAR point clouds and a custom data transformation to KITTI standards.

Experimental results

Research questions

• RQ1 Can a three-stage fusion scheme effectively estimate object counts and improve map fidelity in dynamic environments?
• RQ2 Can federated learning be effectively applied to LiDAR-based feature models in a distributed vehicle network with unknown object counts?
• RQ3 How well can knowledge distillation generate reliable training labels when real labels are missing?
• RQ4 Does the proposed EDFL framework achieve performance close to a fully supervised FL baseline with no labels?

Key findings

• The proposed three-stage fusion method achieved the highest average precision at IoU=0.7 among all compared methods, outperforming max-score and mean fusion.
• The EDFL-based fusion method achieved performance close to the perfect FL baseline, especially in long-range (LR) and high-occlusion (LO) scenarios.
• Even without ground-truth labels, the EDFL framework significantly improved detection accuracy compared to methods without FL or fusion.
• The framework reduced communication overhead by avoiding raw data transfer, relying instead on model parameter aggregation and distilled labels.
• The edge server’s fused map served as an effective teacher model, enabling accurate pseudo-label generation for local training.