[Paper Review] Learning Enhancement From Degradation: A Diffusion Model For Fundus Image Enhancement

Pujin Cheng, Lin Li+4 more|arXiv (Cornell University)|Mar 8, 2023

Retinal Imaging and Analysis7 citations

TL;DR

This paper proposes LED, a novel diffusion model for fundus image enhancement that learns enhancement from a data-driven degradation process, avoiding reliance on handcrafted degradation models. By modeling the distribution of high-quality images and integrating with existing enhancement frameworks, LED achieves state-of-the-art performance in preserving clinically relevant features with minimal information modification, outperforming SOTA methods on multiple metrics including VSD (+10.10), DRA (+1.87), and FMSE reduction (-5.11).

ABSTRACT

The quality of a fundus image can be compromised by numerous factors, many of which are challenging to be appropriately and mathematically modeled. In this paper, we introduce a novel diffusion model based framework, named Learning Enhancement from Degradation (LED), for enhancing fundus images. Specifically, we first adopt a data-driven degradation framework to learn degradation mappings from unpaired high-quality to low-quality images. We then apply a conditional diffusion model to learn the inverse enhancement process in a paired manner. The proposed LED is able to output enhancement results that maintain clinically important features with better clarity. Moreover, in the inference phase, LED can be easily and effectively integrated with any existing fundus image enhancement framework. We evaluate the proposed LED on several downstream tasks with respect to various clinically-relevant metrics, successfully demonstrating its superiority over existing state-of-the-art methods both quantitatively and qualitatively. The source code is available at https://github.com/QtacierP/LED.

Motivation & Objective

• Address the challenge of enhancing low-quality fundus images due to complex, hard-to-model degradations such as glare, blur, and poor illumination.
• Overcome limitations of existing methods that rely on manually designed degradation models or suffer from information modification in unpaired learning.
• Develop a flexible, plug-in enhancement framework compatible with any existing fundus image enhancement method.
• Improve clinical relevance by preserving disease-specific features like vessels and optic cups during enhancement.
• Demonstrate superior performance using both reference-based and non-reference metrics across multiple datasets and downstream tasks.

Proposed method

• Employ a data-driven degradation model to learn realistic degradation mappings from unpaired high-quality to low-quality fundus images.
• Train a conditional diffusion model to learn the inverse enhancement process using paired degraded and high-quality images.
• Integrate the diffusion model into a coarse-to-fine framework: a base enhancement network produces a coarse output, which the diffusion model refines.
• Embed the high-quality image into a Gaussian distribution using the non-noisy low-quality image as an auxiliary input, enabling smooth and continuous enhancement.
• Use a pre-trained quality assessment model to validate that degraded images are closer to real low-quality images than enhanced ones, justifying the training strategy.
• Enable inference integration with any existing enhancement framework by applying the diffusion model as a refinement head.

Experimental results

Research questions

• RQ1 Can a diffusion model trained on data-driven degradation mappings outperform existing fundus image enhancement methods in preserving clinically relevant features?
• RQ2 Does modeling the distribution of high-quality images via diffusion reduce information modification and over-enhancement compared to GAN-based or cycle-consistent methods?
• RQ3 Can the proposed LED framework be effectively and flexibly integrated into existing SOTA enhancement pipelines to boost performance?
• RQ4 How does the LED framework compare to state-of-the-art methods on both reference-based and non-reference metrics across diverse fundus image datasets?
• RQ5 To what extent does LED preserve lesion-specific features such as vessels and optic cups compared to other methods?

Key findings

• LED achieves a 45.39-point improvement in FIQA over the original degraded images, indicating significant enhancement in perceptual quality.
• On the VSD metric, LED improves by 10.10 points compared to the degraded baseline, demonstrating superior preservation of structural details.
• LED reduces FMSE by 5.11 points compared to the original degraded images, indicating better fidelity and reduced distortion.
• When combined with PCENet, LED increases DRA by 24.66 points, showing strong performance on multi-disease datasets and improved lesion detection.
• In vessel segmentation, LED-refined outputs from PCENet and I-SECRET show clearer vessel structures with minimal artifacts, outperforming baseline methods visually and quantitatively.
• The integration of LED with I-SECRET increases VSD by 1.70 points and DRA by 12.14 points, confirming its effectiveness across diverse enhancement pipelines.