[Paper Review] ROS georegistration: Aerial Multi-spectral Image Simulator for the Robot Operating System

Andrew Willis, Kevin Brink+1 more|arXiv (Cornell University)|Jan 19, 2022

Robotics and Sensor-Based Localization4 citations

TL;DR

This paper presents ROSgeoregistration, a ROS/Gazebo-based simulator that integrates Google Earth Engine's multi-spectral imagery with high-fidelity, geospatially and radiometrically accurate simulation of aerial electro-optical (EO) and synthetic aperture radar (SAR) imagery. It enables real-time, synchronized simulation of sensor data (EO/SAR, IMU, GPS) and ground truth pose for testing image-based navigation and georegistration algorithms in a realistic, scalable UAS simulation environment with full integration into the ROS ecosystem.

ABSTRACT

This article describes a software package called ROS georegistration intended for use with the Robot Operating System (ROS) and the Gazebo 3D simulation environment. ROSgeoregistration provides tools for the simulation, test and deployment of aerial georegistration algorithms and is made available with a link provided in the paper. A model creation package is provided which downloads multi-spectral images from the Google Earth Engine database and, if necessary, incorporates these images into a single, possibly very large, reference image. Additionally a Gazebo plugin which uses the real-time sensor pose and image formation model to generate simulated imagery using the specified reference image is provided along with related plugins for UAV relevant data. The novelty of this work is threefold: (1) this is the first system to link the massive multi-spectral imaging database of Google's Earth Engine to the Gazebo simulator, (2) this is the first example of a system that can simulate geospatially and radiometrically accurate imagery from multiple sensor views of the same terrain region, and (3) integration with other UAS tools creates a new holistic UAS simulation environment to support UAS system and subsystem development where real-world testing would generally be prohibitive. Sensed imagery and ground truth registration information is published to client applications which can receive imagery synchronously with telemetry from other payload sensors, e.g., IMU, GPS/GNSS, barometer, and windspeed sensor data. To highlight functionality, we demonstrate ROSgeoregistration for simulating Electro-Optical (EO) and Synthetic Aperture Radar (SAR) image sensors and an example use case for developing and evaluating image-based UAS position feedback, i.e., pose for image-based Guidance Navigation and Control (GNC) applications.

Motivation & Objective

• To address the challenge of limited access to realistic, multi-spectral, georeferenced aerial imagery for GPS-denied navigation research.
• To enable simulation of synchronized, real-time sensor data (EO/SAR, IMU, GPS, barometer) for UAS system and subsystem development.
• To provide a holistic, ROS-integrated simulation environment that supports end-to-end testing of image-based guidance, navigation, and control (GNC) pipelines.
• To bridge the gap between large-scale Earth observation data (Google Earth Engine) and high-fidelity flight simulation (Gazebo/ROS).
• To support the development and evaluation of georegistration algorithms for EO-to-EO and EO-to-SAR image matching.

Proposed method

• The system uses a model creation package to automatically download multi-spectral images from Google Earth Engine and stitch them into a large, georeferenced reference image.
• A custom Gazebo plugin generates real-time, sensor-specific simulated imagery using the reference image and accurate camera/sensor pose and image formation models.
• The simulator supports multiple sensor modalities, including EO and SAR, with radiometric and geometric fidelity to real-world conditions.
• It publishes synchronized telemetry data (IMU, GPS, airspeed, magnetometer) alongside imagery to enable data fusion and ground truth assessment.
• The system enables real-time, non-real-time, and playback-based simulation of complete UAS sensor and navigation pipelines.
• It integrates with existing ROS tools and UAV simulation frameworks to form a cohesive, extensible simulation environment.

Experimental results

Research questions

• RQ1 Can a simulation framework effectively link Google Earth Engine’s multi-spectral database with the Gazebo/ROS ecosystem to enable realistic UAS simulation?
• RQ2 Can geospatially and radiometrically accurate EO and SAR imagery be generated from the same terrain region with consistent pose and sensor modeling?
• RQ3 To what extent can the simulator support the development and evaluation of image-based navigation and georegistration algorithms?
• RQ4 How well do feature-based and mutual information-based registration methods perform on simulated EO-to-EO and EO-to-SAR image pairs?
• RQ5 Can the simulator support realistic, high-altitude UAS flight profiles with synchronized, truth-accurate sensor data for navigation R&D?

Key findings

• The simulator successfully links Google Earth Engine’s multi-spectral database with Gazebo and ROS, enabling automated acquisition and stitching of large-scale reference terrain maps.
• The system generates geospatially and radiometrically accurate EO and SAR imagery from the same reference terrain, with consistent pose and sensor modeling across views.
• Feature-based matching (ORB with RANSAC) performs well for EO-to-EO image registration, achieving accurate homography estimation as verified by ground truth.
• Mutual information-based registration outperforms feature-based methods for EO-to-SAR matching, demonstrating robustness to modality differences.
• The simulator provides synchronized, truth-accurate telemetry (IMU, GPS, etc.) alongside imagery, enabling end-to-end testing of image-aided navigation pipelines.
• The tool enables real-time and non-real-time simulation of complete UAS sensor and navigation systems, supporting both research and development workflows.