[Paper Review] Infrared and visible image fusion using Latent Low-Rank Representation

Hui Li, Xiao‐Jun Wu|arXiv (Cornell University)|Apr 24, 2018

Advanced Image Fusion Techniques31 references126 citations

TL;DR

The paper proposes a fusion method based on latent low-rank representation (LatLRR) that decomposes infrared and visible images into low-rank (global) and salient (local) parts, fuses them with different strategies, and reconstructs a fused image, showing improved performance over several state-of-the-art methods.

ABSTRACT

Infrared and visible image fusion is an important problem in the field of image fusion which has been applied widely in many fields. To better preserve the useful information from source images, in this paper, we propose a novel image fusion method based on latent low-rank representation(LatLRR) which is simple and effective. Firstly, the source images are decomposed into low-rank parts(global structure) and salient parts(local structure) by LatLRR. Then, the low-rank parts are fused by weighted-average strategy to preserve more contour information. Then, the salient parts are simply fused by sum strategy which is a efficient operation in this fusion framework. Finally, the fused image is obtained by combining the fused low-rank part and the fused salient part. Compared with other fusion methods experimentally, the proposed method has better fusion performance than state-of-the-art fusion methods in both subjective and objective evaluation. The Code of our fusion method is available at https://github.com/hli1221/imagefusion\_Infrared\_visible\_latlrr

Motivation & Objective

• Motivate improved preservation of global and local information in infrared/visible fusion.
• Leverage LatLRR to decompose source images into low-rank and salient parts.
• Fuse low-rank parts with a weighted-average strategy and salient parts with a sum strategy.
• Reconstruct the fused image from the fused low-rank and salient components.
• Evaluate the method against several baselines using subjective and objective metrics.

Proposed method

• Decompose each source image into a low-rank part (global structure) and a salient part (local structure) using LatLRR.
• Fuse low-rank parts via a weighted-average scheme (weights = 0.5 and 0.5).
• Fuse salient parts via a simple sum strategy (weights = 1 and 1).
• Reconstruct the final fused image by adding the fused low-rank part and the fused salient part.

Experimental results

Research questions

• RQ1 Can LatLRR-based decomposition preserve both global and local structures in infrared/visible fusion?
• RQ2 Do different fusion strategies for low-rank and salient parts yield improved fusion quality compared to single-strategy methods?
• RQ3 How does the LatLRR-based fusion method perform against state-of-the-art fusion methods on objective metrics and visual quality?

Key findings

• The proposed LatLRR-based method achieves better objective metrics (SCD, SSIM_a) and lower artifacts (N_abf) than several baselines on 21 image pairs.
• Subjectively, the fused results retain more salient features and details with fewer artifacts compared to competing methods.
• Quantitative results show the proposed method achieves the best SCD, SSIM_a, and N_abf values among the tested methods.
• The method provides natural-looking fusion with preserved structure and brightness information.