[论文解读] HINet: Half Instance Normalization Network for Image Restoration

Liangyu Chen, Xin Lu+3 more|arXiv (Cornell University)|May 13, 2021

Advanced Image Processing Techniques参考文献 58被引用 26

一句话总结

tldr: HINet 引入 Half Instance Normalization 块和两阶段 U-Net 架构来提升图像修复任务，在降低 MACs 的同时实现更快推理并达到最先进的 PSNR/SSIM。

ABSTRACT

In this paper, we explore the role of Instance Normalization in low-level vision tasks. Specifically, we present a novel block: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks. Based on HIN Block, we design a simple and powerful multi-stage network named HINet, which consists of two subnetworks. With the help of HIN Block, HINet surpasses the state-of-the-art (SOTA) on various image restoration tasks. For image denoising, we exceed it 0.11dB and 0.28 dB in PSNR on SIDD dataset, with only 7.5% and 30% of its multiplier-accumulator operations (MACs), 6.8 times and 2.9 times speedup respectively. For image deblurring, we get comparable performance with 22.5% of its MACs and 3.3 times speedup on REDS and GoPro datasets. For image deraining, we exceed it by 0.3 dB in PSNR on the average result of multiple datasets with 1.4 times speedup. With HINet, we won 1st place on the NTIRE 2021 Image Deblurring Challenge - Track2. JPEG Artifacts, with a PSNR of 29.70. The code is available at https://github.com/megvii-model/HINet.

研究动机与目标

• Motivate the use of Instance Normalization in low-level image restoration tasks.
• Propose a lightweight normalization-based block (HIN Block) to boost restoration performance.
• Design a two-stage multi-stage network (HINet) leveraging HIN Blocks, cross-stage feature fusion, and supervised attention.
• Demonstrate superior accuracy with lower computational cost across denoising, deblurring, and deraining datasets.

提出的方法

• Introduce Half Instance Normalization Block (HIN Block) that applies Instance Normalization to half of the channels and concatenates with the remaining half.
• Build a two-subnetwork multi-stage architecture (HINet) where each stage is a U-Net using HIN Blocks in the encoder, and ResBlocks with skip connections in the decoder.
• Incorporate Cross-Stage Feature Fusion (CSFF) and a Supervised Attention Module (SAM) between stages to enrich multi-scale features.
• Train end-to-end with a PSNR-based loss across both subnetworks, encouraging residual learning for restoration.

实验结果

研究问题

• RQ1 Can Half Instance Normalization improve feature representation and restoration quality in low-level vision tasks?
• RQ2 Does a two-stage, multi-scale architecture with CSFF and SAM outperform single-stage or prior multi-stage nets in denoising, deblurring, and deraining?
• RQ3 What is the impact of applying IN to only half the channels (HIN) versus full normalization or no normalization?
• RQ4 How does HINet compare to state-of-the-art methods in PSNR/SSIM and efficiency (MACs, runtime) across standard benchmarks?

主要发现

Dataset	Method	PSNR	SSIM	MACs (G)	Time (ms)	Speedup vs MPRNet
SIDD	HINet (ours)	39.99	0.958	170.71	29.8	2.9×
SIDD	MPRNet [56]	39.71	0.958	573.50	100	1×
REDS	HINet	28.83	0.847	170.71	27.0	3.3×
REDS	MPRNet [56]	28.81	0.847	760.11	90.1	1×
GoPro	HINet	32.77	0.959	170.71	27.0	3.3×
GoPro	MPRNet [56]	32.66	0.959	760.11	90.1	1×
Rain13k (average)	HINet	33.03	0.919	170.71	27.0	1.4×
Rain13k (average)	MPRNet [56]	32.73	0.916	141.28	37.4	1×

• HINet achieves state-of-the-art or competitive PSNR/SSIM on denoising (SIDD) and deblurring (REDS, GoPro) with significantly reduced MACs and faster inference compared to the previous best (MPRNet).
• On SIDD denoising, HINet with 7.5% of MACs reaches 39.82 dB PSNR versus 39.71 dB for MPRNet (33% slower).
• On GoPro deblurring, HINet attains 32.77 dB PSNR compared to MPRNet’s 32.66 dB with substantial MACs reduction (22.5% MACs, 3.3× speedup).
• On Rain deraining (Rain13k and related), HINet delivers up to 0.3 dB PSNR improvement over MPRNet with 1.4× speedup on average.
• HIN blocks consistently enhance performance across datasets (e.g., 0.12 dB PSNR gain on REDS with HINet Simple; 0.42 dB gain on GoPro with a different baseline).
• HINet won 1st place in NTIRE 2021 Image Deblurring Challenge Track 2 (JPEG Artifacts).