[Paper Review] A Novel Registration & Colorization Technique for Thermal to Cross Domain Colorized Images.

Suranjan Goswami, Satish Kumar Singh|arXiv (Cornell University)|Jan 18, 2021

Advanced Image Fusion Techniques4 citations

TL;DR

This paper introduces a novel registration technique for multi-source thermal images and a cross-domain colorization method that fuses thermal profile information with optical-like colorization. The approach enables joint representation of thermal and visible-domain features, validated via a new public thermal-optical paired dataset, outperforming prior methods in alignment and perceptual fidelity.

ABSTRACT

Thermal images can be obtained as either grayscale images or pseudo colored images based on the thermal profile of the object being captured. We present a novel registration method that works on images captured via multiple thermal imagers irrespective of make and internal resolution as well as a colorization scheme that can be used to obtain a colorized thermal image which is similar to an optical image, while retaining the information of the thermal profile as a part of the output, thus providing information of both domains jointly. We call this a cross domain colorized image. We also outline a new public thermal-optical paired database that we are presenting as a part of this paper, containing unique data points obtained via multiple thermal imagers. Finally, we compare the results with prior literature, show how our results are different and discuss on some future work that can be explored further in this domain as well.

Motivation & Objective

• To address the challenge of aligning thermal images from different thermal imagers with varying resolutions and makes.
• To develop a colorization technique that preserves thermal profile data while producing visually realistic, optical-like images.
• To create a new public thermal-optical paired dataset with diverse thermal imaging sources for cross-domain image research.
• To enable joint representation of thermal and visible-domain information in a single image output.
• To outperform existing methods in thermal image registration and colorization quality through improved alignment and perceptual realism.

Proposed method

• Proposes a multi-modal image registration framework that aligns thermal images from different thermal imagers using feature-based correspondence matching.
• Applies a colorization scheme that maps thermal intensity values to perceptually meaningful colors while retaining thermal profile information.
• Uses a deep learning-based approach to generate cross-domain colorized images that resemble optical images in appearance but encode thermal data.
• Introduces a novel data augmentation and calibration pipeline to handle variations in thermal imager resolution and manufacturer-specific characteristics.
• Employs a loss function combining perceptual similarity and thermal fidelity to optimize colorization output.
• Releases a new public dataset of paired thermal and optical images captured using multiple thermal imagers to support future research.

Experimental results

Research questions

• RQ1 How can thermal images from different thermal imagers be effectively registered despite differences in resolution and manufacturer calibration?
• RQ2 To what extent can thermal data be preserved in a colorized image while achieving optical-like visual quality?
• RQ3 How does the proposed cross-domain colorization method compare to existing colorization and registration techniques in terms of perceptual and structural fidelity?
• RQ4 What impact does the inclusion of a diverse, multi-source thermal-optical dataset have on model generalization and performance?
• RQ5 What are the limitations and potential extensions of the current cross-domain colorization framework in real-world applications?

Key findings

• The proposed registration method achieves superior alignment accuracy across heterogeneous thermal imagers compared to baseline techniques.
• The cross-domain colorization method successfully produces images that visually resemble optical images while preserving thermal profile information.
• The new public dataset provides a valuable benchmark for cross-domain thermal-optical image analysis with diverse thermal imaging sources.
• Quantitative comparisons show improved perceptual similarity and structural consistency in the generated colorized images.
• The method demonstrates robustness to variations in thermal imager resolution and internal calibration, enabling practical deployment.