[Paper Review] Underwater Image Enhancement via Learning Water Type Desensitized Representations

Zhenqi Fu, Xiaopeng Lin+3 more|arXiv (Cornell University)|Feb 1, 2021

Image Enhancement Techniques4 citations

TL;DR

This paper proposes SCNet, a U-Net-based underwater image enhancement method that learns water type desensitized representations through joint spatial and channel-wise normalization. By whitening activations across spatial dimensions and standardizing/re-injecting first two moments across channels, SCNet achieves state-of-the-art performance in visual quality and generalization across diverse underwater water types without requiring prior knowledge of water type.

ABSTRACT

We present a novel underwater image enhancement method termed SCNet to improve the image quality meanwhile cope with the degradation diversity caused by the water. SCNet is based on normalization schemes across both spatial and channel dimensions with the key idea of learning water type desensitized features. Specifically, we apply whitening to de-correlate activations across spatial dimensions for each instance in a mini-batch. We also eliminate channel-wise correlation by standardizing and re-injecting the first two moments of the activations across channels. The normalization schemes of spatial and channel dimensions are performed at each scale of the U-Net to obtain multi-scale representations. With such water type irrelevant encodings, the decoder can easily reconstruct the clean signal and be unaffected by the distortion types. Experimental results on two real-world underwater image datasets show that our approach can successfully enhance images with diverse water types, and achieves competitive performance in visual quality improvement.

Motivation & Objective

• Address the challenge of degradation diversity in underwater images caused by varying water types (e.g., coastal, deep oceanic, muddy).
• Overcome the limitations of existing methods that rely on prior knowledge of water type or fail to generalize across diverse water conditions.
• Develop a data-driven approach that enhances image quality without requiring explicit water type classification or multiple model variants.
• Improve generalization by learning appearance- and style-irrelevant representations through normalization across spatial and channel dimensions.

Proposed method

• Apply instance whitening to de-correlate activations across spatial dimensions within each mini-batch, reducing spatial-level water-type dependence.
• Standardize and re-inject the first two moments (mean and variance) of activations across channels to eliminate channel-wise correlations.
• Integrate spatial and channel-wise normalization at every scale of a U-Net encoder-decoder architecture to learn multi-scale, water type desensitized features.
• Use a squeeze-and-excitation (SE) module in the final decoder block to enhance feature modeling capacity.
• Train the network using a combined loss: L2 (MSE) loss for pixel-level reconstruction and LPIPS loss for perceptual quality, with λ=0.1 balancing the two.
• Leverage real-world datasets (UIEBD and RUIE) for training and evaluation, using full-reference metrics (SSIM, PSNR, LPIPS) and qualitative analysis.

Experimental results

Research questions

• RQ1 Can a normalization-based approach effectively handle the diversity of underwater image degradations caused by different water types?
• RQ2 Does joint normalization across spatial and channel dimensions lead to more robust, water type-agnostic representations compared to standard normalization or no normalization?
• RQ3 Can a simple U-Net architecture with normalization components outperform complex, task-specific network designs in underwater image enhancement?
• RQ4 How does the proposed method generalize to real-world datasets without reference images, such as RUIE?

Key findings

• SCNet achieves the highest SSIM (0.8625), PSNR (22.0816), and lowest LPIPS (0.1936) on the UIEBD dataset, outperforming six SOTA methods including model-free, prior-based, and data-driven approaches.
• The ablation study confirms that both spatial-wise normalization (SN) and channel-wise normalization (CN) are essential, with the full SCNet model (SN + CN) yielding the best performance.
• SCNet produces visually natural and vivid results across diverse underwater scenes, as shown in qualitative comparisons on both UIEBD and RUIE datasets.
• The method generalizes well to real-world data without reference images, as demonstrated by consistent enhancement quality on the RUIE dataset.
• The performance gain over U-Net baseline is significant: SSIM increases from 0.8350 to 0.8625, and PSNR from 19.5114 to 22.0816, indicating strong improvement in both structural and pixel-level fidelity.
• The use of LPIPS loss with MSE loss improves perceptual quality, as evidenced by lower LPIPS scores and more natural-looking enhanced outputs.