[Paper Review] Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restoration

Zhao Chen, Weiling Cai+2 more|arXiv (Cornell University)|Nov 28, 2023

Image Enhancement Techniques4 citations

TL;DR

This paper proposes WF-Diff, a novel underwater image enhancement framework that leverages wavelet-domain frequency analysis and diffusion models to improve image quality. By decomposing images into low- and high-frequency subbands via discrete wavelet transform, the method isolates degradation components and applies frequency-specific refinement through a plug-and-play Frequency Residual Diffusion Adjustment Module (FRDAM), achieving state-of-the-art performance on real-world underwater datasets.

ABSTRACT

Underwater images are subject to intricate and diverse degradation, inevitably affecting the effectiveness of underwater visual tasks. However, most approaches primarily operate in the raw pixel space of images, which limits the exploration of the frequency characteristics of underwater images, leading to an inadequate utilization of deep models' representational capabilities in producing high-quality images. In this paper, we introduce a novel Underwater Image Enhancement (UIE) framework, named WF-Diff, designed to fully leverage the characteristics of frequency domain information and diffusion models. WF-Diff consists of two detachable networks: Wavelet-based Fourier information interaction network (WFI2-net) and Frequency Residual Diffusion Adjustment Module (FRDAM). With our full exploration of the frequency domain information, WFI2-net aims to achieve preliminary enhancement of frequency information in the wavelet space. Our proposed FRDAM can further refine the high- and low-frequency information of the initial enhanced images, which can be viewed as a plug-and-play universal module to adjust the detail of the underwater images. With the above techniques, our algorithm can show SOTA performance on real-world underwater image datasets, and achieves competitive performance in visual quality.

Motivation & Objective

• To address the limitations of existing underwater image enhancement (UIE) methods that primarily operate in pixel space and underutilize frequency domain characteristics.
• To investigate the distribution of degradation components in the frequency domain, particularly identifying that color distortion is mainly encoded in low-frequency subbands and texture loss in high-frequency subbands.
• To develop a unified framework that integrates frequency domain analysis with diffusion models to enhance both global color fidelity and local texture details.
• To design a plug-and-play module, FRDAM, that enables targeted refinement of high- and low-frequency components in the wavelet domain for improved restoration quality.

Proposed method

• The framework employs a Wavelet-based Fourier Information Interaction Network (WFI2-net) that performs frequency domain interaction in the wavelet space to enhance low- and high-frequency components separately.
• WFI2-net uses a cross-frequency conditioner (CFC) to enable information exchange between low- and high-frequency subbands, improving representation learning.
• The Frequency Residual Diffusion Adjustment Module (FRDAM) applies diffusion-based denoising in the wavelet domain, refining low- and high-frequency components independently to restore fine details and correct color casts.
• The method uses a multi-loss training strategy including L1 reconstruction loss, perceptual loss, and amplitude-aware loss to preserve structural and semantic consistency.
• The diffusion process is applied in the wavelet domain rather than pixel space, allowing focused refinement on specific frequency bands and reducing artifacts from uncontrolled noise sampling.
• The framework is modular: WFI2-net performs initial enhancement, and FRDAM acts as a universal post-processing module for further detail refinement.

Experimental results

Research questions

• RQ1 How are degradation components (color cast, blur, noise) distributed across different frequency bands in underwater images?
• RQ2 Can wavelet-domain frequency analysis improve the representation of degradation patterns compared to pixel-space methods?
• RQ3 Does applying diffusion models in the wavelet domain, rather than pixel space, lead to better restoration quality and reduced artifacts?
• RQ4 Can a plug-and-play frequency-specific diffusion module (FRDAM) effectively refine high- and low-frequency components independently to enhance image details and color fidelity?

Key findings

• The ablation study confirms that all components of WFI2-net—SFFB, WTB, and CFC—are essential, with the full configuration achieving 21.87 PSNR and 0.8622 SSIM on UIEBD.
• The inclusion of all loss functions (Lh, Lrec, La, CFC) in WFI2-net yields the best performance (21.87 PSNR, 0.8622 SSIM), proving their synergistic effect.
• FRDAM significantly improves performance: the full FRDAM (with low- and high-frequency refinement) achieves 23.86 PSNR, outperforming standard diffusion models by over 3 dB.
• Visual results show that FRDAM effectively reduces artifacts and color distortion while enhancing texture details, especially in complex underwater scenes.
• The method achieves state-of-the-art performance on both UIEBD and LSUI datasets, with competitive results in FID (23.55) and LPIPS (0.0820) metrics.
• The wavelet-domain diffusion approach (FRDAM) outperforms pixel-space diffusion, as shown by a 2.5 dB improvement in PSNR when using wavelet-based refinement.