[Paper Review] SCANet: Self-Paced Semi-Curricular Attention Network for Non-Homogeneous Image Dehazing

Yu Guo, Yuan Gao+5 more|arXiv (Cornell University)|Apr 17, 2023

Image Enhancement Techniques4 citations

TL;DR

This paper proposes SCANet, a self-paced semi-curricular attention network for non-homogeneous image dehazing that enhances feature extraction in haze-dense regions through a novel 'attention generation-scene reconstruction' paradigm. By leveraging luminance-aware attention map generation and a self-paced semi-curricular learning strategy, SCANet achieves state-of-the-art performance with improved convergence and reduced learning ambiguity, outperforming existing methods on NTIRE benchmarks in both PSNR and SSIM while maintaining low FLOPs and parameter count.

ABSTRACT

The presence of non-homogeneous haze can cause scene blurring, color distortion, low contrast, and other degradations that obscure texture details. Existing homogeneous dehazing methods struggle to handle the non-uniform distribution of haze in a robust manner. The crucial challenge of non-homogeneous dehazing is to effectively extract the non-uniform distribution features and reconstruct the details of hazy areas with high quality. In this paper, we propose a novel self-paced semi-curricular attention network, called SCANet, for non-homogeneous image dehazing that focuses on enhancing haze-occluded regions. Our approach consists of an attention generator network and a scene reconstruction network. We use the luminance differences of images to restrict the attention map and introduce a self-paced semi-curricular learning strategy to reduce learning ambiguity in the early stages of training. Extensive quantitative and qualitative experiments demonstrate that our SCANet outperforms many state-of-the-art methods. The code is publicly available at https://github.com/gy65896/SCANet.

Motivation & Objective

• To address the challenge of non-homogeneous haze, which causes spatially varying degradation such as blurring, color distortion, and low contrast, particularly in complex scenes.
• To improve the robustness of dehazing models in regions with high haze concentration and significant luminance variation, where existing methods often fail.
• To reduce learning ambiguity and improve model convergence during training by introducing a self-paced semi-curricular learning strategy for attention map generation.
• To develop a lightweight yet effective architecture that balances high-quality detail reconstruction with computational efficiency.

Proposed method

• The method employs a two-branch network: an attention generation network (AGN) to produce attention maps sensitive to non-uniform haze distribution, and a scene reconstruction network (SRN) to restore the haze-free image.
• Luminance differences in the input image are used to constrain the attention map generation, ensuring higher attention is assigned to regions with significant intensity changes.
• A self-paced semi-curricular learning (SCL) strategy is introduced to progressively train the model on easier samples first, reducing early-stage learning ambiguity in multi-objective optimization.
• The network is trained using a multi-loss objective: L1 loss on features ($\mathcal{L}_{sl1}^{f}$), L1 loss on attention maps ($\mathcal{L}_{sl1}^{a}$), perceptual loss ($\mathcal{L}_{p}$), MS-SSIM loss ($\mathcal{L}_{\text{MS-SSIM}}$), and adversarial loss ($\mathcal{L}_{a}$).
• The attention map is supervised via $\mathcal{L}_{sl1}^{a}$ to prevent over-attention on low-importance regions, improving reconstruction fidelity.
• The training process uses ground-truth attention maps ($M_{GT}$) only during training, not at inference, ensuring the model learns to generate attention maps end-to-end.

Experimental results

Research questions

• RQ1 How can a deep learning model effectively capture and exploit the non-uniform distribution of haze in real-world scenes?
• RQ2 What training strategy can reduce learning ambiguity and improve convergence when optimizing for multiple objectives in early training stages?
• RQ3 Can luminance-aware attention supervision enhance the model's ability to reconstruct texture details in high-haze and high-contrast regions?
• RQ4 How does the integration of self-paced semi-curricular learning improve the performance and stability of non-homogeneous dehazing networks?
• RQ5 What is the contribution of each component—attention generation, reconstruction, and loss functions—to the overall dehazing quality and efficiency?

Key findings

• SCANet achieves a PSNR of 20.37 and SSIM of 0.6933 on the combined NTIRE2020, NTIRE2021, and NTIRE2023 test sets, outperforming state-of-the-art methods in both metrics.
• The ablation study shows that adding the attention generation network (AGN) improves PSNR from 18.84 to 19.29, demonstrating its necessity for non-homogeneous dehazing.
• Incorporating the self-paced semi-curricular learning (SCL) strategy increases PSNR from 19.92 to 20.02, indicating improved convergence and performance.
• The full model with all loss functions (including MS-SSIM and adversarial loss) achieves the highest PSNR (20.37) and SSIM (0.6933), confirming the effectiveness of the multi-loss design.
• SCANet achieves a low FLOP count and fewer parameters than comparable models, with inference taking only 0.1962 seconds per 1200×1600 image on an RTX 3080 GPU.
• Qualitative results on the NTIRE2023 test set show that SCANet effectively suppresses haze in dense regions and preserves natural textures and colors, especially in sky and high-luminance areas.