[Paper Review] Restoring Vision in Hazy Weather with Hierarchical Contrastive Learning

Tao Wang, Guangpin Tao+5 more|arXiv (Cornell University)|Dec 22, 2022

Image Enhancement Techniques4 citations

TL;DR

This paper proposes Hierarchical Contrastive Dehazing (HCD), a novel image dehazing method that enhances feature fusion through a hierarchical interaction module and introduces a hierarchical contrastive loss (HCL) to jointly optimize positive and negative sample representations. HCD achieves state-of-the-art performance on SOTS, HazeRD, and DENSE-HAZE datasets, with gains of up to +1.14 dB in PSNR and improved SSIM and LPIPS scores over prior methods.

ABSTRACT

Image restoration under hazy weather condition, which is called single image dehazing, has been of significant interest for various computer vision applications. In recent years, deep learning-based methods have achieved success. However, existing image dehazing methods typically neglect the hierarchy of features in the neural network and fail to exploit their relationships fully. To this end, we propose an effective image dehazing method named Hierarchical Contrastive Dehazing (HCD), which is based on feature fusion and contrastive learning strategies. HCD consists of a hierarchical dehazing network (HDN) and a novel hierarchical contrastive loss (HCL). Specifically, the core design in the HDN is a hierarchical interaction module, which utilizes multi-scale activation to revise the feature responses hierarchically. To cooperate with the training of HDN, we propose HCL which performs contrastive learning on hierarchically paired exemplars, facilitating haze removal. Extensive experiments on public datasets, RESIDE, HazeRD, and DENSE-HAZE, demonstrate that HCD quantitatively outperforms the state-of-the-art methods in terms of PSNR, SSIM and achieves better visual quality.

Motivation & Objective

• To address the limitations of existing deep learning-based dehazing methods that underutilize hierarchical features and lack negative-oriented supervision.
• To improve feature representation by fusing multi-scale features across shallow and deep layers in a CNN-based architecture.
• To enhance model generalization and restoration quality by introducing a novel hierarchical contrastive loss (HCL) that leverages both positive and negative sample pairs.
• To achieve superior dehazing performance in terms of PSNR, SSIM, and perceptual quality on synthetic and real-world datasets.

Proposed method

• The proposed Hierarchical Dehazing Network (HDN) integrates a hierarchical feature extractor, a hierarchical interaction module (HIM), and a multi-output image reconstruction module to enable multi-resolution feature fusion.
• The HIM enables bidirectional feature refinement by propagating high-level semantic features to shallow layers to suppress noise and injecting low-level details into deep layers for richer representation.
• A hierarchical contrastive loss (HCL) is designed to maximize similarity between positive pairs (e.g., clean image and its dehazed version) and dissimilarity between negative pairs (e.g., hazy input and clean image) across multiple levels of the network.
• HCL operates on hierarchically paired exemplars, allowing the model to learn robust representations by contrasting features at different scales.
• The model is trained end-to-end using a combination of reconstruction loss and HCL, improving both pixel-level fidelity and perceptual quality.
• The HCL is shown to be universally effective when transferred to other state-of-the-art networks like GridDehazeNet, FFA-Net, MSBDN, and AECR-Net, improving their PSNR and SSIM scores.

Experimental results

Research questions

• RQ1 Can hierarchical feature fusion across multiple scales improve dehazing performance in terms of PSNR and SSIM?
• RQ2 Does incorporating negative-oriented learning via contrastive loss enhance the model’s ability to distinguish haze components from clean content?
• RQ3 Can a hierarchical contrastive loss strategy improve feature representation in a dehazing network beyond standard supervision?
• RQ4 How does the proposed HCD method compare to state-of-the-art methods on both synthetic and real-world dehazing benchmarks?
• RQ5 Is the proposed HCL loss generalizable across different network architectures?

Key findings

• HCD achieves a PSNR gain of +1.14 dB on the indoor subset of SOTS and +0.45 dB on the outdoor subset, significantly outperforming previous state-of-the-art methods.
• On the HazeRD dataset, HCD achieves a PSNR improvement of +0.83 dB, and on DENSE-HAZE, it improves by +0.61 dB compared to existing methods.
• In perceptual metrics, HCD achieves the lowest LPIPS score of 0.0416 on SOTS, indicating superior perceptual quality compared to baselines.
• When HCL is applied to existing networks like GridDehazeNet and FFA-Net, it boosts PSNR by up to 1.08 dB and SSIM by 0.0051, demonstrating its generalization capability.
• Ablation studies confirm that HCL significantly improves test PSNR even without accelerating convergence, proving its effectiveness in representation learning.
• Visual comparisons show that HCD produces clearer, brighter images with fewer artifacts than state-of-the-art methods like MPR and OSFD.