[Paper Review] GAP-net for Snapshot Compressive Imaging

Ziyi Meng, Shirin Jalali+1 more|arXiv (Cornell University)|Dec 13, 2020

Sparse and Compressive Sensing Techniques55 references74 citations

TL;DR

GAP-net unrolls the generalized alternating projection framework for snapshot compressive imaging by inserting trainable CNN denoisers at each stage, achieving competitive video and spectral SCI performance with convergence guarantees.

ABSTRACT

Snapshot compressive imaging (SCI) systems aim to capture high-dimensional ($\ge3$D) images in a single shot using 2D detectors. SCI devices include two main parts: a hardware encoder and a software decoder. The hardware encoder typically consists of an (optical) imaging system designed to capture {compressed measurements}. The software decoder on the other hand refers to a reconstruction algorithm that retrieves the desired high-dimensional signal from those measurements. In this paper, using deep unfolding ideas, we propose an SCI recovery algorithm, namely GAP-net, which unfolds the generalized alternating projection (GAP) algorithm. At each stage, GAP-net passes its current estimate of the desired signal through a trained convolutional neural network (CNN). The CNN operates as a denoiser that projects the estimate back to the desired signal space. For the GAP-net that employs trained auto-encoder-based denoisers, we prove a probabilistic global convergence result. Finally, we investigate the performance of GAP-net in solving video SCI and spectral SCI problems. In both cases, GAP-net demonstrates competitive performance on both synthetic and real data. In addition to having high accuracy and high speed, we show that GAP-net is flexible with respect to signal modulation implying that a trained GAP-net decoder can be applied in different systems. Our code is at https://github.com/mengziyi64/ADMM-net.

Motivation & Objective

• Develop a unified SCI reconstruction method applicable to both video SCI and spectral SCI.
• Incorporate a CNN-based denoiser at each stage of a GAP-based projection scheme.
• Provide theoretical convergence guarantees for an AE-based GAP-net.
• Demonstrate competitive accuracy and speed on synthetic and real SCI data.

Proposed method

• Formulate SCI reconstruction as minimizing (1/2)||y-Hx||^2 + tau*Omega(x).
• Use GAP to iteratively project onto the linear model y=Hx with an auxiliary variable v.
• At each stage, perform a Euclidean projection x^(k+1)=v^(k)+H^T(HH^T)^{-1}(y-Hv^(k)).
• Apply a trained CNN denoiser D_{k+1} to map x^(k+1) back into signal space, producing v^(k+1)=D_{k+1}(x^(k+1)).
• Train stage-specific denoisers (AE, DnCNN, U-net, ResNet) for video and spectral SCI.
• Provide a convergence analysis showing global convergence of AE-based GAP-net under prescribed conditions.

Experimental results

Research questions

• RQ1 Can GAP-net provide accurate reconstructions for both video SCI and spectral SCI within a single framework?
• RQ2 Do stage-wise CNN denoisers improve reconstruction quality while maintaining computational efficiency?
• RQ3 What are the convergence guarantees for AE-based GAP-net in snapshot compressive imaging scenarios?
• RQ4 How does GAP-net compare to other iterative, end-to-end, and PnP methods on synthetic and real SCI data?

Key findings

• GAP-net achieves competitive PSNR/SSIM with significantly faster runtimes than DeSCI and many alternatives.
• Using 9 stages with U-net or ResNet denoisers yields higher quality reconstructions than several baselines on six synthetic video SCI datasets (Kobe, Traffic, Runner, Drop, Vehicle, Aerial).
• GAP-net with 9 stages and a fast CNN denoiser attains ~4 ms per measurement when using ResNet, enabling near real-time reconstruction.
• On real video SCI data, GAP-net produces sharper motion details and cleaner backgrounds than several baselines.
• For spectral SCI, GAP-net with 9 stages and U-net denoiser outperforms multiple iterative and learning-based methods in PSNR/SSIM across 10 scenes (synthetic data) and shows strong qualitative results on real data.
• Theoretical results establish global convergence of AE-based GAP-net under the chosen generative model and sensing assumptions.