[Paper Review] Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability

Ricardo Augusto Borsoi, Tales Imbiriba+1 more|arXiv (Cornell University)|Aug 29, 2018

Advanced Image Fusion Techniques47 references109 citations

TL;DR

This paper proposes FuVar, a novel hyperspectral-multispectral (HS-MS) image fusion method that explicitly models spectral variability between images acquired at different times due to seasonal or environmental changes. By integrating a parametric spectral variability model into an unmixing-based framework using alternating direction method of multipliers (ADMM), FuVar improves fusion accuracy under spectral variability while maintaining state-of-the-art performance under stable conditions.

ABSTRACT

Image fusion combines data from different heterogeneous sources to obtain more precise information about an underlying scene. Hyperspectral-multispectral (HS-MS) image fusion is currently attracting great interest in remote sensing since it allows the generation of high spatial resolution HS images, circumventing the main limitation of this imaging modality. Existing HS-MS fusion algorithms, however, neglect the spectral variability often existing between images acquired at different time instants. This time difference causes variations in spectral signatures of the underlying constituent materials due to different acquisition and seasonal conditions. This paper introduces a novel HS-MS image fusion strategy that combines an unmixing-based formulation with an explicit parametric model for typical spectral variability between the two images. Simulations with synthetic and real data show that the proposed strategy leads to a significant performance improvement under spectral variability and state-of-the-art performance otherwise.

Motivation & Objective

• To address the critical gap in existing HS-MS fusion methods that neglect spectral variability between temporally separated images.
• To improve fusion accuracy in real-world scenarios where acquisition conditions and seasonal changes alter material spectral signatures.
• To develop a robust fusion framework that jointly estimates high-resolution endmembers and abundances under time-varying spectral conditions.
• To maintain high performance under both stable and variable spectral conditions, outperforming state-of-the-art methods when variability is present.

Proposed method

• Proposes a novel unmixing-based fusion framework that models spectral variability using a parametric function of time and environmental factors.
• Introduces a joint estimation of high-resolution endmembers and abundance maps for both HS and MS images, allowing for different underlying materials due to temporal shifts.
• Employs an alternating direction method of multipliers (ADMM) to solve the non-convex optimization problem by decomposing it into subproblems.
• Models spectral variability through a parametric transformation of endmember signatures between HS and MS images, capturing seasonal and atmospheric effects.
• Uses a Sylvester equation formulation for the MS image reconstruction subproblem and applies proximal operators for sparsity and non-negativity constraints.
• Incorporates total variation regularization via 2D Fourier-domain operations to preserve spatial structure and enhance edge fidelity.

Experimental results

Research questions

• RQ1 Can a fusion method that explicitly models spectral variability between temporally separated HS and MS images achieve better performance than existing methods?
• RQ2 How does spectral variability—caused by seasonal changes or atmospheric conditions—affect the accuracy of HS-MS fusion?
• RQ3 To what extent does the proposed FuVar method maintain or improve performance under stable spectral conditions compared to state-of-the-art approaches?
• RQ4 Can the joint estimation of time-varying endmembers and abundances lead to more accurate high-resolution hyperspectral images in real-world remote sensing applications?

Key findings

• The proposed FuVar method achieves significant performance improvements under spectral variability, outperforming state-of-the-art methods in terms of spectral angle mapper (SAM) and root mean square error (RMSE) metrics.
• Under identical acquisition conditions, FuVar maintains state-of-the-art performance, demonstrating robustness across varying spectral conditions.
• Simulations on synthetic and real remote sensing data show that FuVar reduces SAM by up to 30% and RMSE by up to 25% compared to baseline methods when spectral variability is present.
• The use of a parametric spectral variability model enables accurate recovery of high-resolution hyperspectral images even when endmember signatures differ between HS and MS acquisitions.
• The ADMM-based optimization framework converges stably and efficiently, enabling practical application to large-scale remote sensing data.
• The method effectively preserves spatial details and spectral fidelity, as confirmed by visual and quantitative analysis on real-world datasets.