[Paper Review] A Generic Fundus Image Enhancement Network Boosted by Frequency Self-supervised Representation Learning

Heng Li, Haofeng Liu+6 more|PubMed|Sep 2, 2023

Retinal Imaging and Analysis44 references9 citations

TL;DR

GFE-Net enhances degraded fundus images without supervised data by learning structure-aware representations from frequency-based self-supervision and seamlessly coupling them with image enhancement. It achieves robust, generalizable results across datasets with improved deployment efficiency.

ABSTRACT

Fundus photography is prone to suffer from image quality degradation that impacts clinical examination performed by ophthalmologists or intelligent systems. Though enhancement algorithms have been developed to promote fundus observation on degraded images, high data demands and limited applicability hinder their clinical deployment. To circumvent this bottleneck, a generic fundus image enhancement network (GFE-Net) is developed in this study to robustly correct unknown fundus images without supervised or extra data. Levering image frequency information, self-supervised representation learning is conducted to learn robust structure-aware representations from degraded images. Then with a seamless architecture that couples representation learning and image enhancement, GFE-Net can accurately correct fundus images and meanwhile preserve retinal structures. Comprehensive experiments are implemented to demonstrate the effectiveness and advantages of GFE-Net. Compared with state-of-the-art algorithms, GFE-Net achieves superior performance in data dependency, enhancement performance, deployment efficiency, and scale generalizability. Follow-up fundus image analysis is also facilitated by GFE-Net, whose modules are respectively verified to be effective for image enhancement.

Motivation & Objective

• Address the challenge of enhancing unknown degraded fundus images without supervised or test-data access.
• Learn robust, structure-preserving representations from degraded images using frequency information.
• Couple representation learning with image enhancement in a seamless architecture to avoid adversarial training.
• Demonstrate data efficiency, generalizability, and deployment practicality across multiple datasets.

Proposed method

• Introduce frequency self-supervised representation learning (SSRL) to learn structure-aware representations from degraded fundus views.
• Synthesize degraded views from clear fundus images to train SSRL without extra annotations.
• Use high-frequency maps (HFM) reconstruction as the SSRL pretext to preserve retinal structures.
• Couple a shared encoder with two decoders: one for representation learning (D_R) and one for image enhancement (D_E).
• Optimize with a composite loss L_total = L_R + L_E + L_cyc to train end-to-end without adversarial training.
• Leverage a U-Net-like architecture and explicit objective functions to ensure convergence and stability.

Experimental results

Research questions

• RQ1 Can a generic fundus image enhancement model correct unknown degraded images using only synthesized data?
• RQ2 Does frequency-based self-supervised learning preserve retinal structures better than traditional supervised or adversarial approaches?
• RQ3 How does integrating SSRL with enhancement affect data dependency, deployment efficiency, and scale generalizability?
• RQ4 What is the performance of GFE-Net across datasets degraded by imaging interference and cataracts?

Key findings

• GFE-Net achieves superior enhancement performance while reducing data dependency compared to state-of-the-art baselines.
• The frequency self-supervised learning component preserves retinal structures during enhancement.
• Training convergence is efficient and avoids adversarial training pitfalls, with competitive deployment costs.
• GFE-Net demonstrates strong generalizability across datasets with different degradation types (imaging interferences and cataracts).
• The model trained on synthesized data can be applied to multiple datasets without test-data access or extra tuning.