[Paper Review] Towards Adversarial Retinal Image Synthesis

Pedro Costa, Adrián Galdrán+5 more|arXiv (Cornell University)|Jan 31, 2017

Retinal Imaging and AnalysisMedicine16 references76 citations

TL;DR

The paper learns to synthesize retinal fundus images from binary vessel trees using an image-to-image translation framework with adversarial learning, trained on vessel-vessel-tree paired data generated via U-Net vessel segmentation.

ABSTRACT

Synthesizing images of the eye fundus is a challenging task that has been previously approached by formulating complex models of the anatomy of the eye. New images can then be generated by sampling a suitable parameter space. In this work, we propose a method that learns to synthesize eye fundus images directly from data. For that, we pair true eye fundus images with their respective vessel trees, by means of a vessel segmentation technique. These pairs are then used to learn a mapping from a binary vessel tree to a new retinal image. For this purpose, we use a recent image-to-image translation technique, based on the idea of adversarial learning. Experimental results show that the original and the generated images are visually different in terms of their global appearance, in spite of sharing the same vessel tree. Additionally, a quantitative quality analysis of the synthetic retinal images confirms that the produced images retain a high proportion of the true image set quality.

Motivation & Objective

• Motivate synthetic retinal data generation to augment limited annotated medical datasets.
• Propose a data-driven pipeline that maps binary retinal vessel trees to realistic retinal images.
• Leverage adversarial learning to produce high-quality synthetic fundus images while preserving vessel structure.

Proposed method

• Pair real retinal images with binary vessel trees obtained via a U-Net segmentation model.
• Use an image-to-image translation framework with a generator G and discriminator D to map vessel trees to retinal images.
• Combine adversarial loss with a local L1 loss to produce sharp yet globally consistent images: L(G,D)=L_adv(G,D)+λ E[||r−G(v)||1].
• Train D to classify image patches as real vs synthetic and train G to fool D while minimizing the L1 term.
• Upgrade the generator architecture to handle 512x512 inputs/outputs by adding processing layers in both downsampling and upsampling paths.

Experimental results

Research questions

• RQ1 Can retinal images be plausibly synthesized from binary vessel trees while preserving global structure and realism?
• RQ2 How close are synthetic retinal images to real ones under standard no-reference quality metrics?
• RQ3 What is the impact of using a no-reference vessel-focused metric versus a global image-structure metric for evaluating synthetic retinal images?
• RQ4 What failure modes arise when vessel tree information is incomplete or degraded by segmentation errors?

Key findings

• Synthetic retinal images can be produced from binary vessel trees that are visually different in color/illumination but share global vascular geometry with real images.
• Quantitative evaluation shows real and synthetic images have similar ISC scores (0.9872±0.0468 vs 0.9889±0.0398; p>0.05 after artifact removal).
• Qv scores are higher for real images than synthetic ones (0.1254±0.0340 vs 0.1047±0.0136; p<0.05).
• Artifacts occur in about 6 of 177 test cases due to undercomplete vessel trees from segmentation failures, indicating dependence on vessel tree quality.