[Paper Review] Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

Yuanhao Cai, Jing Lin+6 more|arXiv (Cornell University)|May 20, 2022

Sparse and Compressive Sensing Techniques63 citations

TL;DR

Introduces a degradation-aware unfolding framework (DAUF) with a novel Half-Shuffle Transformer (HST) denoiser, yielding Transformer-based deep unfolding for CASSI-based HSI reconstruction that achieves state-of-the-art performance with lower computation and memory costs.

ABSTRACT

In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement. Among these algorithms, deep unfolding methods demonstrate promising performance but suffer from two issues. Firstly, they do not estimate the degradation patterns and ill-posedness degree from the highly related CASSI to guide the iterative learning. Secondly, they are mainly CNN-based, showing limitations in capturing long-range dependencies. In this paper, we propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. Moreover, we customize a novel Half-Shuffle Transformer (HST) that simultaneously captures local contents and non-local dependencies. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST), for HSI reconstruction. Experiments show that DAUHST significantly surpasses state-of-the-art methods while requiring cheaper computational and memory costs. Code and models will be released at https://github.com/caiyuanhao1998/MST

Motivation & Objective

• Motivate improved hyperspectral image reconstruction in coded aperture snapshot spectral imaging (CASSI) by explicitly estimating degradation and ill-posedness from the compressed measurements.
• Propose a principled MAP-based unfolding framework (DAUF) that adapts iterations using degradation-aware parameters.
• Design a novel Half-Shuffle Transformer (HST) to serve as an efficient denoiser that captures local and non-local dependencies.
• Integrate DAUF with HST to form the DAUHST method and demonstrate superior performance and efficiency over state-of-the-art methods.

Proposed method

• Formulate the CASSI degradation model as y = Φx + n and cast HSI reconstruction as MAP optimization with a prior R(x).
• Apply half-quadratic splitting to decouple x and z, leading to iterative updates with a closed-form x-update leveraging the Sherman–Mittag–Wishart-like inversion adapted to ΦΦ^T being diagonal.
• Introduce iteration-specific degradation parameters α and β estimated from y and Φ to modulate the linear projection and the denoising strength in each stage.
• Develop a Half-Shuffle Transformer (HST) denoiser with HS-MSA that combines local-window self-attention and non-local cross-window interactions via token shuffling, enabling global-like dependencies at lower cost than full global attention.
• Embed HST inside each iteration of DAUF, enabling end-to-end training while preserving interpretability tied to the imaging model.

Experimental results

Research questions

• RQ1 How can one estimate degradation patterns and ill-posedness degree from compressed CASSI measurements to guide iterative reconstruction?
• RQ2 Can a Transformer-based denoiser outperform CNN-based priors in deep unfolding for HSI reconstruction under CASSI?
• RQ3 What is the effect of incorporating degradation-aware parameter estimation on reconstruction quality and efficiency?
• RQ4 Does the Half-Shuffle Transformer provide a favorable accuracy-cost trade-off compared to standard global or local Transformers in this task?

Key findings

• DAUHST substantially outperforms state-of-the-art unfolding methods across simulation scenes, achieving large PSNR/SSIM gains at similar or lower computational cost.
• The degradation-aware parameter estimator (E) effectively informs each iteration about the CASSI system’s degradation patterns, guiding adaptive scaling of the linear projection and denoising strength.
• The Half-Shuffle Transformer (HST) with HS-MSA better captures local and non-local dependencies than alternative MSAs, contributing to notable improvements in both quantitative metrics and visual quality.
• DAUHST demonstrates strong robustness to noise and real-data conditions, delivering clearer HSIs with richer spectral fidelity in both simulated and real SCI experiments.