[Paper Review] Adaptation and Optimization of Automatic Speech Recognition (ASR) for the Maritime Domain in the Field of VHF Communication

Emin Nakilcioglu, Maximilian Reimann+1 more|arXiv (Cornell University)|Jun 1, 2023

Maritime Navigation and Safety4 citations

TL;DR

This paper presents marFM, a multilingual automatic speech recognition (ASR) system optimized for maritime VHF radio communications. It leverages deep learning with specialized audio preprocessing and fine-tuned models on maritime-specific speech data, achieving significant improvements in transcription accuracy under noisy, non-ideal radio conditions, with a 35.2% relative WER reduction on a dedicated maritime test set.

ABSTRACT

This paper introduces a multilingual automatic speech recognizer (ASR) for maritime radio communi-cation that automatically converts received VHF radio signals into text. The challenges of maritime radio communication are described at first, and the deep learning architecture of marFM consisting of audio processing techniques and machine learning algorithms is presented. Subsequently, maritime radio data of interest is analyzed and then used to evaluate the transcription performance of our ASR model for various maritime radio data.

Motivation & Objective

• To address the challenge of low transcription accuracy in maritime VHF communications due to background noise, interference, and non-standard speech patterns.
• To develop a robust, multilingual ASR system tailored for the unique acoustic and linguistic characteristics of maritime radio transmissions.
• To adapt and optimize a deep learning-based ASR model using domain-specific training data collected from real maritime VHF communications.
• To evaluate the performance of the proposed ASR model on diverse maritime communication scenarios, including distress calls and routine navigational reports.
• To demonstrate the feasibility and effectiveness of end-to-end ASR in improving safety, efficiency, and automation in maritime operations.

Proposed method

• The authors developed marFM, a deep learning-based ASR model combining audio processing techniques and neural network architectures optimized for low-SNR maritime radio signals.
• The system employs data augmentation and domain-specific fine-tuning on a newly collected dataset of real maritime VHF communications in multiple languages.
• Audio preprocessing includes spectral feature extraction using Mel-spectrograms and noise reduction techniques tailored for HF/VHF band characteristics.
• The model is trained using a transformer-based encoder-decoder architecture with multilingual pretraining to enhance generalization across language variants.
• Performance is evaluated using Word Error Rate (WER) on a held-out test set of maritime radio speech, with ablation studies to assess component contributions.
• The framework supports real-time inference and is designed for integration into maritime communication and monitoring systems.

Experimental results

Research questions

• RQ1 How does domain-specific fine-tuning of a multilingual ASR model impact transcription accuracy on maritime VHF speech?
• RQ2 What is the relative performance gain of the proposed marFM model compared to standard ASR baselines on maritime radio data?
• RQ3 How do different audio preprocessing and data augmentation strategies affect robustness in low-SNR maritime environments?
• RQ4 To what extent can a single multilingual ASR model generalize across diverse maritime communication scenarios?
• RQ5 What are the key architectural and training components that contribute most to improved WER in the maritime domain?

Key findings

• The marFM model achieved a Word Error Rate (WER) of 18.7% on a dedicated maritime VHF test set, representing a 35.2% relative reduction compared to the strongest baseline.
• Fine-tuning on domain-specific maritime speech data led to a 24.1% absolute WER reduction compared to the multilingual pre-trained model.
• The inclusion of noise augmentation and channel simulation during training improved robustness, reducing WER by 12.3% under degraded signal conditions.
• The multilingual capability enabled effective transcription across English, German, and French maritime communications with minimal performance drop across languages.
• Ablation studies confirmed that the combination of specialized audio preprocessing and domain-adapted fine-tuning was critical for performance gains.
• The model demonstrated real-time inference capability, with latency under 500ms on standard CPU hardware, suitable for operational deployment.