[Paper Review] Spacecraft Pose Estimation Based on Unsupervised Domain Adaptation and on a 3D-Guided Loss Combination

Juan Ignacio Bravo Pérez-Villar, Álvaro García‐Martín+1 more|arXiv (Cornell University)|Dec 27, 2022

Space Satellite Systems and Control4 citations

TL;DR

This paper proposes a 3D-guided loss combination and unsupervised domain adaptation (UDA) method for spacecraft pose estimation, leveraging synthetic data to address domain shift between simulated and real-world test conditions. The approach achieves second place in the 2021 ESA-Stanford Pose Estimation Challenge with the lowest average error across both Sunlamp and Lightbox test sets, demonstrating robustness to illumination variations and reflections.

ABSTRACT

Spacecraft pose estimation is a key task to enable space missions in which two spacecrafts must navigate around each other. Current state-of-the-art algorithms for pose estimation employ data-driven techniques. However, there is an absence of real training data for spacecraft imaged in space conditions due to the costs and difficulties associated with the space environment. This has motivated the introduction of 3D data simulators, solving the issue of data availability but introducing a large gap between the training (source) and test (target) domains. We explore a method that incorporates 3D structure into the spacecraft pose estimation pipeline to provide robustness to intensity domain shift and we present an algorithm for unsupervised domain adaptation with robust pseudo-labelling. Our solution has ranked second in the two categories of the 2021 Pose Estimation Challenge organised by the European Space Agency and the Stanford University, achieving the lowest average error over the two categories.

Motivation & Objective

• Address the lack of real in-orbit spacecraft pose data for training data-driven models.
• Overcome the domain shift between synthetic training data and real test data due to differences in illumination, reflections, and background conditions.
• Improve pose estimation robustness under challenging space conditions such as strong reflections and variable lighting.
• Develop an unsupervised domain adaptation strategy that generates reliable pseudo-labels without ground-truth annotations on target data.
• Achieve state-of-the-art performance on the 2021 Spacecraft Pose Estimation Challenge (SPEC2021) across two distinct real-world test scenarios.

Proposed method

• Integrate 3D structure information into the loss function via a 3D-guided loss that combines 2D heatmap loss with 3D pose alignment loss, improving generalization under intensity domain shifts.
• Use a two-stage pipeline: first, predict 2D keypoint heatmaps via a CNN (e.g., HRNet), then estimate 6-DoF pose using PnP from 2D-3D correspondences.
• Apply unsupervised domain adaptation via iterative pseudo-labelling using RANSAC-EPnP to generate reliable pose predictions on unlabeled target images.
• Implement a self-training loop where the model is retrained on high-confidence pseudo-labeled target samples, with dynamic reprojection error thresholds to control label quality.
• Use a validation-based early stopping criterion and confidence filtering to avoid noise accumulation during pseudo-labelling.
• Train on synthetic SPEED+ dataset (60k images) and fine-tune on target datasets (Sunlamp and Lightbox) using UDA with adaptive thresholding.

Experimental results

Research questions

• RQ1 Can incorporating 3D geometric structure into the loss function improve pose estimation robustness under domain shift caused by illumination and reflection differences?
• RQ2 How effective is unsupervised domain adaptation with robust pseudo-labelling for spacecraft pose estimation when no real target labels are available?
• RQ3 Does dynamically adjusting the reprojection error threshold during pseudo-labelling improve performance on visually diverse test sets like Sunlamp and Lightbox?
• RQ4 Can a single model trained on synthetic data generalize effectively to real-world test scenarios with strong reflections and complex backgrounds?
• RQ5 What is the impact of 3D-guided loss on reducing position and orientation errors in challenging space conditions?

Key findings

• The proposed method achieved the second-lowest average error across both Sunlamp and Lightbox test sets in the 2021 SPEC2021 Challenge, with the best overall average score.
• On the Sunlamp dataset, the method achieved a total error of 1.0692, significantly lower than the baseline of 1.2324.
• On the Lightbox dataset, the method achieved a total error of 1.0546, outperforming the baseline of 1.1823.
• The use of a 3D-guided loss improved generalization, reducing errors on target domains despite the domain shift from synthetic to real data.
• Iterative pseudo-labelling with adaptive reprojection error (from 2.0 to 1.0) led to a slower but more stable increase in pseudo-label count on the Lightbox set, though final performance was comparable to fixed thresholding.
• The method maintained high performance on both datasets, indicating strong robustness to varying illumination and background complexity.