[Paper Review] A New Wave in Robotics: Survey on Recent mmWave Radar Applications in Robotics

Kyle Harlow, Hyesu Jang+3 more|arXiv (Cornell University)|May 2, 2023

Advanced SAR Imaging Techniques4 citations

TL;DR

This survey explores the growing role of mmWave radar in robotics, highlighting its advantages over cameras and lidars in adverse conditions like fog and dust. It reviews applications in localization, mapping, object detection, and semantic segmentation, emphasizing FMCW radar's robustness and potential for fusion or standalone use in challenging environments.

ABSTRACT

We survey the current state of millimeterwave (mmWave) radar applications in robotics with a focus on unique capabilities, and discuss future opportunities based on the state of the art. Frequency Modulated Continuous Wave (FMCW) mmWave radars operating in the 76--81GHz range are an appealing alternative to lidars, cameras and other sensors operating in the near visual spectrum. Radar has been made more widely available in new packaging classes, more convenient for robotics and its longer wavelengths have the ability to bypass visual clutter such as fog, dust, and smoke. We begin by covering radar principles as they relate to robotics. We then review the relevant new research across a broad spectrum of robotics applications beginning with motion estimation, localization, and mapping. We then cover object detection and classification, and then close with an analysis of current datasets and calibration techniques that provide entry points into radar research.

Motivation & Objective

• To analyze the current state of mmWave radar applications in robotics, focusing on its unique advantages over traditional sensors like cameras and lidars.
• To identify key robotics applications where mmWave radar excels, particularly in visually degraded environments such as fog, dust, and smoke.
• To review recent algorithmic advances in radar-based localization, mapping, object detection, and semantic segmentation.
• To highlight open research challenges, including 6DoF localization, 3D radar mapping, online calibration, and semantic labeling beyond vehicles and pedestrians.
• To advocate for the development of standardized benchmarks and datasets to accelerate radar research in robotics, similar to KITTI for cameras and lidar.

Proposed method

• Surveying recent literature on mmWave radar applications in robotics, with a focus on FMCW radar operating in the 76–81 GHz band.
• Analyzing radar principles relevant to robotics, including range, Doppler velocity, and angular resolution, and their implications for perception tasks.
• Categorizing and reviewing radar applications across motion estimation, localization, mapping, object detection, and semantic classification.
• Evaluating existing datasets (e.g., Boreas) and calibration techniques, including both offline and potential online calibration methods.
• Comparing radar performance to cameras and lidars in terms of data density, resolution, robustness to visual degradation, and operational range.
• Proposing future research directions, such as panoptic radar segmentation, fusion with neural radiance fields, and improved 3D mapping for planning.

Experimental results

Research questions

• RQ1 How does mmWave radar outperform cameras and lidars in visually degraded environments such as fog, dust, and smoke?
• RQ2 What are the key algorithmic challenges and recent advances in radar-based localization and mapping, particularly in 6DoF and long-term robustness?
• RQ3 To what extent can mmWave radar be used for semantic perception, including object classification and panoptic segmentation?
• RQ4 What are the open challenges in radar sensor calibration, especially online intrinsic calibration for range and velocity?
• RQ5 How can the full radar waveform be leveraged beyond point clouds to enable new capabilities such as behind-wall sensing or enhanced 3D reconstruction?

Key findings

• mmWave radar provides robust performance in visually degraded environments where cameras and lidars fail, due to its longer wavelengths that penetrate fog, dust, and smoke.
• FMCW mmWave radars operating at 76–81 GHz offer high resolution and long-range detection, with dense data generation suitable for motion estimation and mapping.
• Radar-based localization systems currently achieve ~10 cm error, which lags behind lidar’s ~5 cm, indicating room for improvement through better sensors and algorithms.
• While 2D radar localization is established, 6DoF radar localization remains an open challenge, especially for high-speed UAVs and dynamic environments.
• Semantic radar perception is emerging, with early work on classifying vehicles and pedestrians, but broader labeling for indoor environments like homes and hospitals remains underexplored.
• The Boreas dataset and similar benchmarks are critical for advancing radar odometry and localization, and their adoption is expected to accelerate research in the field.