[Paper Review] Robust SAR ATR on MSTAR with Deep Learning Models trained on Full Synthetic MOCEM data

Benjamin Camus, Corentin Le Barbu+1 more|arXiv (Cornell University)|Jun 15, 2022

Advanced SAR Imaging Techniques4 citations

TL;DR

This paper proposes a robust SAR ATR framework using deep learning models trained on a full synthetic dataset generated with the MOCEM simulator, which significantly differs from real MSTAR measurements. By combining domain randomization and adversarial training, the method achieves 75% accuracy on real-world test data—demonstrating superior generalization over state-of-the-art approaches under realistic, non-ideal conditions.

ABSTRACT

The promising potential of Deep Learning for Automatic Target Recognition (ATR) on Synthetic Aperture Radar (SAR) images vanishes when considering the complexity of collecting training datasets measurements. Simulation can overcome this issue by producing synthetic training datasets. However, because of the limited representativeness of simulation, models trained in a classical way with synthetic images have limited generalization abilities when dealing with real measurement at test time. Previous works identified a set of equally promising deep-learning algorithms to tackle this issue. However, these approaches have been evaluated in a very favorable scenario with a synthetic training dataset that overfits the ground truth of the measured test data. In this work, we study the ATR problem outside of this ideal condition, which is unlikely to occur in real operational contexts. Our contribution is threefold. (1) Using the MOCEM simulator (developed by SCALIAN DS for the French MoD/DGA), we produce a synthetic MSTAR training dataset that differs significantly from the real measurements. (2) We experimentally demonstrate the limits of the state-of-the-art. (3) We show that domain randomization techniques and adversarial training can be combined to overcome this issue. We demonstrate that this approach is more robust than the state-of-the-art, with an accuracy of 75 %, while having a limited impact on computing performance during training.

Motivation & Objective

• To address the generalization gap between synthetic training data and real SAR measurements in ATR systems.
• To evaluate state-of-the-art deep learning models under realistic, non-ideal conditions where synthetic data does not closely match real test data.
• To improve robustness of SAR ATR models by combining domain randomization and adversarial training techniques.
• To demonstrate that synthetic data from a full MOCEM simulation can yield effective generalization when combined with advanced domain adaptation strategies.

Proposed method

• The authors use the MOCEM simulator to generate a comprehensive synthetic MSTAR training dataset that differs significantly from real measurements.
• They apply domain randomization to augment the synthetic data by varying simulation parameters such as noise, aspect angle, and signal characteristics.
• Adversarial training is employed to align the feature distributions of synthetic and real data using a domain discriminator.
• A deep neural network (e.g., CNN-based) is trained end-to-end on the randomized synthetic data to classify SAR targets.
• The training process includes a domain adversarial loss to minimize domain shift between synthetic and real data distributions.
• The model is evaluated on the real MSTAR test set to measure generalization performance.

Experimental results

Research questions

• RQ1 Can deep learning models trained on fully synthetic SAR data generalize effectively to real-world MSTAR test data when the synthetic data does not closely resemble real measurements?
• RQ2 How do domain randomization and adversarial training compare in improving robustness of SAR ATR models under domain shift?
• RQ3 What is the performance of the proposed method in terms of accuracy and computational efficiency on real MSTAR data?
• RQ4 Does combining domain randomization with adversarial training yield better generalization than either technique alone?

Key findings

• The proposed method achieves 75% accuracy on the real MSTAR test set, outperforming state-of-the-art approaches under realistic domain shift conditions.
• Domain randomization alone fails to close the domain gap when the synthetic data is significantly different from real measurements.
• Adversarial training alone provides limited improvement, but when combined with domain randomization, it significantly enhances robustness.
• The combination of domain randomization and adversarial training results in a model that generalizes well to real-world SAR data with minimal computational overhead.