[Paper Review] NaRPA: Navigation and Rendering Pipeline for Astronautics

Roshan T. Eapen, Ramchander Rao Bhaskara+1 more|arXiv (Cornell University)|Nov 3, 2022

Satellite Image Processing and Photogrammetry4 citations

TL;DR

NaRPA is a ray-tracing-based physics engine for aerospace simulations that models global illumination, atmospheric effects, and sensor outputs like images, depth maps, and point clouds. It enables accurate, differentiable rendering for vision-based navigation, demonstrating high-fidelity 3D pose estimation via stereo triangulation and gradient-based optimization in ISS docking simulations.

ABSTRACT

This paper presents Navigation and Rendering Pipeline for Astronautics (NaRPA) - a novel ray-tracing-based computer graphics engine to model and simulate light transport for space-borne imaging. NaRPA incorporates lighting models with attention to atmospheric and shading effects for the synthesis of space-to-space and ground-to-space virtual observations. In addition to image rendering, the engine also possesses point cloud, depth, and contour map generation capabilities to simulate passive and active vision-based sensors and to facilitate the designing, testing, or verification of visual navigation algorithms. Physically based rendering capabilities of NaRPA and the efficacy of the proposed rendering algorithm are demonstrated using applications in representative space-based environments. A key demonstration includes NaRPA as a tool for generating stereo imagery and application in 3D coordinate estimation using triangulation. Another prominent application of NaRPA includes a novel differentiable rendering approach for image-based attitude estimation is proposed to highlight the efficacy of the NaRPA engine for simulating vision-based navigation and guidance operations.

Motivation & Objective

• To develop a physically accurate, ray-traced simulation platform for space-based imaging and navigation.
• To address limitations in optical flow and vision-based navigation caused by inaccurate illumination modeling.
• To enable high-fidelity simulation of space-to-space and ground-to-space observations with atmospheric and shading effects.
• To support the design, testing, and verification of visual navigation algorithms using synthetic sensor data.
• To enable differentiable rendering for end-to-end optimization of 3D pose estimation from 2D images.

Proposed method

• NaRPA employs a ray-tracing pipeline based on the rendering equation to simulate global illumination and light transport in space environments.
• It incorporates atmospheric scattering models, including Rayleigh and Mie scattering, to simulate time-of-day and turbidity effects on light propagation.
• The framework generates multi-modal sensor outputs: photorealistic images, depth maps, contour maps, and point clouds from 3D scene geometry.
• A differentiable rendering module propagates gradients through the rendering pipeline to enable optimization of pose parameters from 2D image observations.
• Pose estimation is performed via nonlinear optimization using the Levenberg–Marquardt algorithm on the residual between synthetic and observed images.
• The system uses a chain-rule-based Jacobian computation to link image formation models with 3D pose parameters for gradient-based refinement.

Experimental results

Research questions

• RQ1 Can a ray-traced rendering engine accurately simulate space-based observations with realistic atmospheric and shading effects?
• RQ2 How well can NaRPA generate synthetic stereo imagery for 3D coordinate estimation using triangulation?
• RQ3 To what extent can differentiable rendering in NaRPA enable accurate 3D pose estimation from 2D image projections?
• RQ4 Can NaRPA serve as a ground-truth simulation environment for testing vision-based navigation algorithms in autonomous spacecraft missions?
• RQ5 How does the inclusion of physically based lighting and atmospheric modeling improve the robustness of optical flow and pose estimation algorithms?

Key findings

• NaRPA successfully generated photorealistic images with accurate atmospheric scattering, including time-of-day and turbidity variations, enhancing realism for space mission simulation.
• The stereo image generation capability of NaRPA enabled accurate 3D coordinate estimation using triangulation, with point cloud data serving as ground truth for validation.
• The differentiable rendering module achieved convergence in pose estimation for an ISS docking maneuver simulation, with cost function decreasing over iterations as shown in Fig. 22.
• The estimated poses from synthetic images closely matched the true poses, visually validated by image comparison across five frames in the simulation sequence.
• The framework demonstrated the feasibility of using gradient-based optimization through the rendering pipeline to infer 3D pose from 2D image observations.
• NaRPA’s multi-sensor output capability supports comprehensive testing of visual navigation algorithms, including those relying on depth and contour data.