[Paper Review] Assessing the Value of Transfer Learning Metrics for RF Domain Adaptation

Lauren J. Wong, Sean McPherson+1 more|arXiv (Cornell University)|Jun 16, 2022

Wireless Signal Modulation Classification4 citations

TL;DR

This paper evaluates transferability metrics LEEP and LogME for radio frequency machine learning (RFML) domain adaptation, demonstrating they predict post-transfer accuracy in CNN-based modulation classifiers with high correlation. The study shows head re-training outperforms fine-tuning and provides guidelines for selecting source models based on channel similarity, enabling efficient and accurate RFML adaptation under data-scarce conditions.

ABSTRACT

The use of transfer learning (TL) techniques has become common practice in fields such as computer vision (CV) and natural language processing (NLP). Leveraging prior knowledge gained from data with different distributions, TL offers higher performance and reduced training time, but has yet to be fully utilized in applications of machine learning (ML) and deep learning (DL) techniques to applications related to wireless communications, a field loosely termed radio frequency machine learning (RFML). This work begins this examination by evaluating the how radio frequency (RF) domain changes encourage or prevent the transfer of features learned by convolutional neural network (CNN)-based automatic modulation classifiers. Additionally, we examine existing transferability metrics, Log Expected Empirical Prediction (LEEP) and Logarithm of Maximum Evidence (LogME), as a means to both select source models for RF domain adaptation and predict post-transfer accuracy without further training.

Motivation & Objective

• To assess the utility of transferability metrics LEEP and LogME in predicting performance for RF domain adaptation in RFML.
• To investigate how RF domain shifts—specifically SNR and frequency offset—impact transfer learning performance in CNN-based modulation classifiers.
• To compare the effectiveness of head re-training versus fine-tuning for RF domain adaptation under limited data.
• To identify key factors influencing successful transfer learning in RFML, such as source-target domain similarity and platform differences.
• To establish practical guidelines for deploying transfer learning in real-world RFML systems with dynamic channel conditions.

Proposed method

• Trained a CNN-based automatic modulation classifier on synthetic RF data with controlled variations in SNR and frequency offset.
• Applied transfer learning via two strategies: head re-training (randomly initialized head, frozen feature extractor) and fine-tuning (fine-tuned full network).
• Computed LEEP and LogME transferability metrics for each source model to predict post-transfer accuracy without additional training.
• Evaluated performance across three experimental sweeps: SNR-only, frequency offset-only, and combined SNR+FO variations.
• Used Spearman’s rank correlation to assess agreement between predicted and actual post-transfer top-1 accuracy for LEEP and LogME.
• Analyzed the relative importance of source-target similarity in channel environment versus hardware platform for transfer success.

Experimental results

Research questions

• RQ1 How well do LEEP and LogME transferability metrics correlate with actual post-transfer accuracy in RF domain adaptation?
• RQ2 Which transfer learning strategy—head re-training or fine-tuning—yields better performance and efficiency in RFML?
• RQ3 How do variations in SNR and frequency offset affect the transferability of learned features in RFML?
• RQ4 Does similarity in channel environment matter more than similarity in hardware platforms when selecting source models?
• RQ5 Can transferability metrics reliably predict post-transfer accuracy without further training in RFML settings?

Key findings

• LEEP and LogME showed strong correlation with actual post-transfer accuracy, with Spearman’s rho values of 0.7496 and 0.7468 for fine-tuning across SNR+FO sweeps.
• Head re-training achieved higher performance and was more computationally efficient than fine-tuning, with a 0.6175 agreement rate in over/under prediction of accuracy.
• Transferring from a more challenging source domain (e.g., lower SNR or higher FO) than the target domain yielded better performance than transferring from an easier domain.
• Source-target similarity in channel environment (e.g., SNR and FO) was a more critical factor for successful transfer than similarity in hardware platforms.
• Combining LEEP and LogME improved prediction reliability, suggesting that ensemble use of metrics enhances robustness in RFML transfer learning.
• The study provides actionable guidelines: prioritize source models with similar channel conditions, prefer head re-training for speed and stability, and use LEEP/LogME for pre-selection without retraining.