[Paper Review] MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space

Jiangwei Weng, Zhiqiang Yan+4 more|arXiv (Cornell University)|May 25, 2024

Advanced Optical Sensing Technologies4 citations

TL;DR

MambaLLIE proposes a global-then-local state space model for low light image enhancement, integrating a Local-Enhanced State Space Module (LESSM) for local 2D dependency preservation and an Implicit Retinex-aware Selective Kernel (IRSK) for adaptive illumination modulation. The method achieves state-of-the-art performance, with a PSNR of 30.12 and SSIM of 0.900 on benchmark datasets, outperforming existing CNN and Transformer-based approaches.

ABSTRACT

Recent advances in low light image enhancement have been dominated by Retinex-based learning framework, leveraging convolutional neural networks (CNNs) and Transformers. However, the vanilla Retinex theory primarily addresses global illumination degradation and neglects local issues such as noise and blur in dark conditions. Moreover, CNNs and Transformers struggle to capture global degradation due to their limited receptive fields. While state space models (SSMs) have shown promise in the long-sequence modeling, they face challenges in combining local invariants and global context in visual data. In this paper, we introduce MambaLLIE, an implicit Retinex-aware low light enhancer featuring a global-then-local state space design. We first propose a Local-Enhanced State Space Module (LESSM) that incorporates an augmented local bias within a 2D selective scan mechanism, enhancing the original SSMs by preserving local 2D dependency. Additionally, an Implicit Retinex-aware Selective Kernel module (IRSK) dynamically selects features using spatially-varying operations, adapting to varying inputs through an adaptive kernel selection process. Our Global-then-Local State Space Block (GLSSB) integrates LESSM and IRSK with LayerNorm as its core. This design enables MambaLLIE to achieve comprehensive global long-range modeling and flexible local feature aggregation. Extensive experiments demonstrate that MambaLLIE significantly outperforms state-of-the-art CNN and Transformer-based methods. Project Page: https://mamballie.github.io/anon/

Motivation & Objective

• To address the limitations of CNNs and Transformers in modeling long-range global degradation and local noise/blurring in low-light images.
• To overcome the inability of standard state space models (SSMs) to effectively capture local visual dependencies while maintaining global long-range modeling.
• To introduce a novel implicit Retinex-aware framework that avoids explicit decomposition of illumination and reflectance, enabling flexible, end-to-end learning.
• To design a Global-then-Local State Space Block (GLSSB) that unifies local enhancement and global context modeling for comprehensive low-light recovery.

Proposed method

• Introduces the Local-Enhanced State Space Module (LESSM), which augments the 2D selective scan mechanism with a local bias to preserve spatial dependency in visual features.
• Proposes the Implicit Retinex-aware Selective Kernel (IRSK), which dynamically selects spatially-varying kernels to adaptively modulate illumination strength through deeper neural representations.
• Designs the Global-then-Local State Space Block (GLSSB) by integrating LESSM and IRSK with LayerNorm, enabling both global long-range modeling and flexible local feature aggregation.
• Employs a two-stage strategy: first modeling global degradation via SSMs, then refining local details using selective kernel operations with implicit Retinex guidance.
• Uses a residual learning scheme with skip connections to stabilize training and improve feature propagation across the network.
• Applies effective normalization and kernel size scheduling, placing smaller kernels at earlier layers and larger ones at deeper layers to avoid edge artifacts from padding.

Experimental results

Research questions

• RQ1 Can a global-then-local state space architecture effectively model both long-range global illumination and local image degradation in low-light images?
• RQ2 How does implicit Retinex-aware feature selection compare to explicit decomposition in terms of performance and generalization?
• RQ3 To what extent does the integration of LESSM and IRSK improve local detail recovery and global consistency compared to vanilla SSMs or standard CNNs/Transformers?
• RQ4 Does the proposed selective kernel mechanism with adaptive kernel selection lead to better illumination modulation than fixed or learnable kernel configurations?

Key findings

• MambaLLIE achieves a PSNR of 30.12 and SSIM of 0.900 on the SIDD benchmark, outperforming state-of-the-art methods including CNNs and Transformers.
• The ablation study shows that removing LESSM reduces PSNR by 0.29 dB compared to the full model, indicating its importance in local modeling.
• The IRSK component alone improves PSNR by 0.96 dB over Baseline-1 and 0.74 dB over Baseline-2, demonstrating its effectiveness in illumination adaptation.
• The implicit Retinex-aware framework improves PSNR by 1.25 dB over an end-to-end baseline without prior guidance, confirming the benefit of prior integration.
• The selective kernel behavior visualization confirms that the model learns to prioritize similar local features over global dependencies, enhancing local consistency.
• The 3×5 kernel configuration achieves the best performance (PSNR: 30.12, SSIM: 0.900), outperforming larger or symmetric kernels, validating the design choice of asymmetric, size-increasing kernels.