[Paper Review] Hyperspectral-Multispectral Image Fusion with Weighted LASSO

Nguyen Tran, Rupali Mankar+2 more|arXiv (Cornell University)|Mar 15, 2020

Advanced Image Fusion Techniques36 references4 citations

TL;DR

This paper proposes a weighted LASSO-based hyperspectral-multispectral image fusion method using ADMM for efficiency and MAP initialization to enhance reconstruction quality. It achieves superior performance in noise reduction, spectral fidelity, and computational speed compared to state-of-the-art methods, particularly under high noise and blur conditions.

ABSTRACT

Spectral imaging enables spatially-resolved identification of materials in remote sensing, biomedicine, and astronomy. However, acquisition times require balancing spectral and spatial resolution with signal-to-noise. Hyperspectral imaging provides superior material specificity, while multispectral images are faster to collect at greater fidelity. We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. The proposed optimization leverages the least absolute shrinkage and selection operator (LASSO) to perform variable selection and regularization. Computational time is reduced by applying the alternating direction method of multipliers (ADMM), as well as initializing the fusion image by estimating it using maximum a posteriori (MAP) based on Hardie's method. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images. Finally, we show how the proposed method can be practically applied in biomedical infrared spectroscopic microscopy.

Motivation & Objective

• To address the trade-off between spectral resolution, spatial resolution, and signal-to-noise ratio in hyperspectral and multispectral imaging.
• To develop a robust fusion method that preserves high spatial resolution and molecular specificity despite noise and blur in input images.
• To reduce computational cost while maintaining high-fidelity reconstruction using efficient optimization techniques.
• To improve spectral fidelity and noise suppression in fused images, especially under challenging imaging conditions.
• To demonstrate practical applicability in biomedical infrared spectroscopic microscopy.

Proposed method

• The method employs a weighted LASSO optimization framework to minimize differences between the fused image and input hyperspectral (HSI) and multispectral (MSI) images using the Frobenius norm.
• It applies nuclear norm regularization via LASSO to enforce sparsity and reduce noise in the fused output.
• The alternating direction method of multipliers (ADMM) is used to decompose the optimization problem, significantly improving computational efficiency.
• The fused image is initialized using a Maximum a Posteriori (MAP) estimate based on Hardie’s method to accelerate convergence and improve accuracy.
• The fusion model integrates modality-specific characteristics such as spectral sampling, spatial resolution, and noise profiles from HSI and MSI inputs.
• The approach is evaluated using standard fusion metrics including PSNR, SAM, ERGAS, and SSIM across multiple public datasets under varying noise conditions.

Experimental results

Research questions

• RQ1 Can a LASSO-based optimization framework effectively fuse hyperspectral and multispectral images while preserving spectral and spatial fidelity?
• RQ2 How does the integration of ADMM and MAP initialization improve computational efficiency and reconstruction quality?
• RQ3 To what extent does the proposed method outperform existing state-of-the-art fusion techniques under high noise and blur?
• RQ4 How does noise distribution (Gaussian vs. Poisson) affect the performance of the fusion method and baseline approaches?
• RQ5 Can the method be practically applied to biomedical infrared spectroscopic microscopy with real-world imaging constraints?

Key findings

• The proposed method achieved a 354.2-second fusion time and 479.3-second algorithm time, significantly outperforming Bayes Sparse, which required 2170.9 and 6976.9 seconds, respectively.
• In the Pavia Center dataset with Poisson noise, the proposed method achieved an average SAM of 3.25°, compared to 6.46° for Bayes Sparse, indicating superior spectral fidelity.
• Under Gaussian noise, the proposed method outperformed Bayes Sparse by an average of 6.2 dB in PSNR across the KSC dataset, with a 4 dB advantage in Indian Pines and Salinas datasets.
• When HSI was corrupted with Poisson noise, the proposed method lost only 6.66 dB in PSNR on average, compared to 1.86 dB for Bayes Sparse, demonstrating greater robustness to noise type.
• In the Pavia University dataset, the proposed method maintained consistent performance across all SNR levels, with a 4 dB PSNR advantage over Bayes Sparse at 50 dB SNR.
• The method showed minimal performance degradation in the KSC dataset, a sparsely textured scene, indicating strong generalization across diverse data types.