[Paper Review] Enabling Intelligent Vehicular Networks Through Distributed Learning in the Non-Terrestrial Networks 6G Vision

David Naseh, Swapnil Sadashiv Shinde+1 more|arXiv (Cornell University)|Sep 7, 2023

Privacy-Preserving Technologies in DataComputer Science3 citations

TL;DR

This paper proposes Federated Split Transfer Learning (FSTL), a novel distributed learning framework integrating Federated Learning (FL), Split Learning (SL), and Transfer Learning (TL) to enable efficient, low-latency, and privacy-preserving machine learning in 6G-enabled Intelligent Vehicular Networks (IVNs). By leveraging High-Altitude Platforms (HAPs) as non-terrestrial network (NTN) infrastructure, FSTL reduces communication overhead and computation load while maintaining high model accuracy even under data heterogeneity and limited resources, outperforming FL, SL, and FSL in accuracy and latency under increasing user loads.

ABSTRACT

The forthcoming 6G-enabled Intelligent Transportation System (ITS) is set to redefine conventional transportation networks with advanced intelligent services and applications. These technologies, including edge computing, Machine Learning (ML), and network softwarization, pose stringent requirements for latency, energy efficiency, and user data security. Distributed Learning (DL), such as Federated Learning (FL), is essential to meet these demands by distributing the learning process at the network edge. However, traditional FL approaches often require substantial resources for satisfactory learning performance. In contrast, Transfer Learning (TL) and Split Learning (SL) have shown effectiveness in enhancing learning efficiency in resource-constrained wireless scenarios like ITS. Non-terrestrial Networks (NTNs) have recently acquired a central place in the 6G vision, especially for boosting the coverage, capacity, and resilience of traditional terrestrial facilities. Air-based NTN layers, such as High Altitude Platforms (HAPs), can have added advantages in terms of reduced transmission distances and flexible deployments and thus can be exploited to enable intelligent solutions for latency-critical vehicular scenarios. With this motivation, in this work, we introduce the concept of Federated Split Transfer Learning (FSTL) in joint air-ground networks for resource-constrained vehicular scenarios. Simulations carried out in vehicular scenarios validate the efficacy of FSTL on HAPs in NTN, demonstrating significant improvements in addressing the demands of ITS applications.

Motivation & Objective

• To address the high latency, communication overhead, and resource constraints in 6G-enabled Intelligent Transportation Systems (ITS) with distributed machine learning.
• To overcome the limitations of traditional Federated Learning (FL), such as high computational load and privacy risks, in resource-constrained vehicular nodes.
• To integrate Transfer Learning (TL) and Split Learning (SL) into FL to enhance model convergence speed and reduce communication costs in vehicular scenarios.
• To evaluate the performance of the proposed FSTL framework in non-terrestrial networks (NTNs), particularly using High-Altitude Platforms (HAPs), for improved coverage and resilience.
• To demonstrate that FSTL maintains high accuracy and low latency even with increasing numbers of vehicular users (VUs) and heterogeneous data distributions.

Proposed method

• Proposes Federated Split Transfer Learning (FSTL), a hybrid framework combining FL, SL, and TL to distribute model training across vehicular users (VUs) and a central server via HAPs.
• Uses a pre-trained model from a source task as a knowledge base to initialize client models, reducing training time and improving convergence in data-scarce environments.
• Applies Split Learning (SL) by dividing the neural network into client-side and server-side components, where only the activation of the cut layer is communicated, minimizing bandwidth usage.
• Integrates Federated Learning (FL) by aggregating gradients from client-side models at the server, enabling global model updates without sharing raw data.
• Employs a hierarchical training process: clients train on local data using a pre-trained model, send only the final layer gradients to the HAP-based server, which aggregates them and updates the global model.
• Utilizes High-Altitude Platforms (HAPs) as relay nodes in NTN to reduce transmission delay, enhance coverage, and support scalable, low-latency communication in vehicular networks.

Experimental results

Research questions

• RQ1 How does FSTL improve learning efficiency and reduce latency compared to FL, SL, and FSL in resource-constrained vehicular networks?
• RQ2 To what extent does the integration of Transfer Learning (TL) with FL and SL enhance model accuracy under data heterogeneity and limited local data?
• RQ3 How does the use of High-Altitude Platforms (HAPs) as NTN infrastructure impact the scalability and performance of distributed learning in vehicular scenarios?
• RQ4 What is the effect of increasing the number of vehicular users (VUs) on the accuracy and latency of FSTL compared to baseline methods?
• RQ5 Can FSTL maintain high performance under model drift and domain shift when local data distributions vary significantly across VUs?

Key findings

• FSTL maintains consistent testing accuracy across different numbers of vehicular users (VUs), with minimal degradation even when data per user is reduced, unlike FL, which shows significant accuracy drops under the same conditions.
• With increasing numbers of VUs, FSTL achieves significantly lower overall latency than SL, FL, and FSL, due to reduced communication overhead and parallel aggregation in the training process.
• FSTL reduces communication cost by transmitting only the gradients of the final (cut) layer from clients to the server, resulting in lower data volume and faster transmission compared to full model parameter exchange in FL.
• The use of a pre-trained model in FSTL enables faster convergence and better performance in data-scarce scenarios, reducing the need for large local datasets on resource-constrained VUs.
• FSTL demonstrates robustness to data heterogeneity, maintaining high accuracy even when local datasets vary in size and quality, due to the knowledge transfer from the pre-trained model.
• Simulations show that FSTL outperforms FL, SL, and FSL in both accuracy and latency, especially as the number of participating VUs increases, confirming its scalability and efficiency in large-scale ITS deployments.