[Paper Review] Generation of Realistic Synthetic Raw Radar Data for Automated Driving Applications using Generative Adversarial Networks

Eduardo C. Fidelis, Fabio Reway+6 more|arXiv (Cornell University)|Aug 4, 2023

Autonomous Vehicle Technology and SafetyEngineering3 citations

TL;DR

This paper proposes a GAN-based method for generating realistic synthetic raw FMCW radar data, using distance and noise as input to generate 16 simultaneous chirps. The approach achieves a low FID score (0.51) and produces realistic Range-Azimuth maps and object detection results, significantly reducing the simulation-to-reality gap compared to traditional ray-tracing methods.

ABSTRACT

The main approaches for simulating FMCW radar are based on ray tracing, which is usually computationally intensive and do not account for background noise. This work proposes a faster method for FMCW radar simulation capable of generating synthetic raw radar data using generative adversarial networks (GAN). The code and pre-trained weights are open-source and available on GitHub. This method generates 16 simultaneous chirps, which allows the generated data to be used for the further development of algorithms for processing radar data (filtering and clustering). This can increase the potential for data augmentation, e.g., by generating data in non-existent or safety-critical scenarios that are not reproducible in real life. In this work, the GAN was trained with radar measurements of a motorcycle and used to generate synthetic raw radar data of a motorcycle traveling in a straight line. For generating this data, the distance of the motorcycle and Gaussian noise are used as input to the neural network. The synthetic generated radar chirps were evaluated using the Frechet Inception Distance (FID). Then, the Range-Azimuth (RA) map is calculated twice: first, based on synthetic data using this GAN and, second, based on real data. Based on these RA maps, an algorithm with adaptive threshold and edge detection is used for object detection. The results have shown that the data is realistic in terms of coherent radar reflections of the motorcycle and background noise based on the comparison of chirps, the RA maps and the object detection results. Thus, the proposed method in this work has shown to minimize the simulation-to-reality gap for the generation of radar data.

Motivation & Objective

• To address the high computational cost and limited realism of traditional ray-tracing methods for FMCW radar simulation.
• To reduce the simulation-to-reality gap in automated driving sensor simulation by generating synthetic radar data that closely mimics real-world measurements.
• To enable efficient data augmentation for radar processing algorithms by generating synthetic data for rare or safety-critical scenarios.
• To develop a data-driven, computationally efficient alternative to physical sensor models for radar simulation.
• To validate the realism of generated radar data through FID scores, RA map comparison, and object detection performance.

Proposed method

• A conditional GAN is trained on real raw IF signals from an FMCW radar, with distance and Gaussian noise as conditioning inputs.
• The generator network uses a WaveGAN-like architecture with batch normalization and ReLU activations, and outputs 16 chirps per forward pass.
• The discriminator is conditioned on both the generated data and the input distance, with feature matching using a pre-trained regression model for distance prediction.
• Input data (IF signals and distance) are scaled using StandardScaler to match the -1 to 1 output range of the generator’s tanh activation.
• The model is trained for 4,655 epochs, with evaluation using FID and nearest-neighbor (NN) distance metrics to assess realism and generalization.
• Range-Azimuth (RA) maps are computed from both real and generated data, and an adaptive threshold + edge detection algorithm is applied for object detection.

Experimental results

Research questions

• RQ1 Can a GAN-based model generate synthetic raw FMCW radar data that is perceptually and statistically similar to real radar measurements?
• RQ2 How does the FID score of the generated data compare to real data and training data, indicating distributional similarity?
• RQ3 To what extent do the generated RA maps and object detection results match those from real radar data?
• RQ4 Can the model generate realistic background noise and coherent radar reflections without copying training samples?
• RQ5 How does the inference speed of the GAN compare to traditional ray-tracing methods in terms of computational efficiency?

Key findings

• The FID score between generated and real data was 0.51, indicating strong distributional similarity and high realism of the synthetic data.
• The nearest-neighbor (NN) distance between generated and training samples was 2.37, higher than training-train (1.00) and comparable to test-train (2.83), suggesting the samples are not simple copies.
• The mean absolute error in motorcycle distance prediction from generated data was 1.4 meters, indicating accurate object localization.
• The generator produced 6,000 chirps in 3.92 seconds (0.65 ms per sample), demonstrating high computational efficiency.
• The RA maps generated from synthetic data closely matched real-world maps in structure and reflection patterns, especially for the motorcycle target.
• Object detection using adaptive thresholding and edge detection achieved consistent results on both real and synthetic data, confirming functional realism.