[Paper Review] MetaUE: Model-based Meta-learning for Underwater Image Enhancement

Zhenwei Zhang, Haorui Yan+2 more|arXiv (Cornell University)|Mar 12, 2023

Image Enhancement Techniques7 citations

TL;DR

This paper proposes MetaUE, a model-based meta-learning framework for underwater image enhancement that jointly estimates clean images, transmission maps, and background light using a physics-inspired image formation model. By pre-training on synthetic underwater data via meta-learning and fine-tuning on real datasets, MetaUE achieves superior generalization and performance across diverse underwater conditions, outperforming state-of-the-art methods in PSNR and SSIM on UIEB and EUVP benchmarks.

ABSTRACT

The challenges in recovering underwater images are the presence of diverse degradation factors and the lack of ground truth images. Although synthetic underwater image pairs can be used to overcome the problem of inadequately observing data, it may result in over-fitting and enhancement degradation. This paper proposes a model-based deep learning method for restoring clean images under various underwater scenarios, which exhibits good interpretability and generalization ability. More specifically, we build up a multi-variable convolutional neural network model to estimate the clean image, background light and transmission map, respectively. An efficient loss function is also designed to closely integrate the variables based on the underwater image model. The meta-learning strategy is used to obtain a pre-trained model on the synthetic underwater dataset, which contains different types of degradation to cover the various underwater environments. The pre-trained model is then fine-tuned on real underwater datasets to obtain a reliable underwater image enhancement model, called MetaUE. Numerical experiments demonstrate that the pre-trained model has good generalization ability, allowing it to remove the color degradation for various underwater attenuation images such as blue, green and yellow, etc. The fine-tuning makes the model able to adapt to different underwater datasets, the enhancement results of which outperform the state-of-the-art underwater image restoration methods. All our codes and data are available at \url{https://github.com/Duanlab123/MetaUE}.

Motivation & Objective

• To address the challenge of limited real underwater image datasets with diverse degradation factors.
• To improve generalization across unknown underwater environments where existing deep learning models often fail.
• To integrate physical underwater image formation models with meta-learning for better interpretability and robustness.
• To develop a pre-trained model that generalizes across various underwater scenarios and can be fine-tuned for specific real-world datasets.

Proposed method

• The method employs a multi-variable convolutional neural network to estimate three components: the clean image, background light, and transmission map, based on the physical underwater image formation model.
• An efficient loss function is designed to tightly integrate the three estimated variables using the physical model constraints.
• Meta-learning is applied to pre-train the model on a diverse synthetic underwater dataset to learn shared prior knowledge across degradation types.
• The pre-trained model is fine-tuned on real underwater datasets to adapt to specific environmental characteristics.
• The architecture uses a UNet-based backbone with separate heads for each variable, enabling joint optimization with physical consistency.
• The model is trained end-to-end using a combination of perceptual, adversarial, and reconstruction losses to improve visual quality and realism.

Experimental results

Research questions

• RQ1 Can a model-based meta-learning approach generalize effectively across diverse underwater image degradation types such as blue, green, and yellow attenuation?
• RQ2 How does pre-training on synthetic data with meta-learning improve zero-shot and few-shot adaptation to real underwater datasets?
• RQ3 To what extent does incorporating the physical underwater image formation model enhance interpretability and performance compared to purely data-driven methods?
• RQ4 Does the joint estimation of clean image, transmission map, and background light lead to better restoration than independent or simplified estimation?
• RQ5 Can the pre-trained model generalize to non-underwater degradation types like haze, low light, and sandstorms?

Key findings

• MetaUE achieved a PSNR of 22.01 and SSIM of 0.8609 on the UIEB dataset, outperforming the baseline Plain UNet (17.54 PSNR, 0.8125 SSIM) and Meta UNet (21.12 PSNR, 0.8054 SSIM).
• On the EUVP dataset, MetaUE achieved 22.12 PSNR and 0.8322 SSIM, surpassing both the Plain UNet (18.76 PSNR, 0.8239 SSIM) and Meta UNet (21.24 PSNR, 0.8211 SSIM).
• The pre-trained model demonstrated strong zero-shot generalization, effectively removing color degradation and restoring illumination in unseen underwater scenarios.
• Fine-tuning significantly improved realism and adaptation to real-world underwater datasets, confirming the effectiveness of domain-specific adaptation.
• The model generalized well to non-underwater degradations such as haze, low light, and sandstorms, outperforming the Rank1 prior method from [4] in visual and quantitative results.
• Visual and residual analysis confirmed accurate estimation of background light and transmission maps, with minimal error between ground truth and predicted clean images.