[Paper Review] Interpretation of Disease Evidence for Medical Images Using Adversarial Deformation Fields

Ricardo Bigolin Lanfredi, Joyce Schroeder+2 more|arXiv (Cornell University)|Jul 3, 2020

Cell Image Analysis Techniques41 references8 citations

TL;DR

This paper proposes DeFI-GAN, a novel method that uses adversarially trained generative networks to produce deformation fields that transform diseased medical images to resemble healthy ones, thereby revealing spatial disease evidence. The method outperforms baseline approaches in longitudinal image validation and uncovers clinically relevant biomarkers and potential dataset biases not detectable with traditional methods.

ABSTRACT

The high complexity of deep learning models is associated with the difficulty of explaining what evidence they recognize as correlating with specific disease labels. This information is critical for building trust in models and finding their biases. Until now, automated deep learning visualization solutions have identified regions of images used by classifiers, but these solutions are too coarse, too noisy, or have a limited representation of the way images can change. We propose a novel method for formulating and presenting spatial explanations of disease evidence, called deformation field interpretation with generative adversarial networks (DeFI-GAN). An adversarially trained generator produces deformation fields that modify images of diseased patients to resemble images of healthy patients. We validate the method studying chronic obstructive pulmonary disease (COPD) evidence in chest x-rays (CXRs) and Alzheimer's disease (AD) evidence in brain MRIs. When extracting disease evidence in longitudinal data, we show compelling results against a baseline producing difference maps. DeFI-GAN also highlights disease biomarkers not found by previous methods and potential biases that may help in investigations of the dataset and of the adopted learning methods.

Motivation & Objective

• To address the lack of fine-grained, spatially coherent explanations for disease evidence in deep learning models for medical imaging.
• To overcome limitations of existing visual attribution methods, which are often too coarse, noisy, or fail to represent anatomical shape and size changes caused by disease.
• To develop a method that generates realistic, spatially meaningful deformations to reveal how models identify disease, particularly morphological changes.
• To validate the method on chronic obstructive pulmonary disease (COPD) in chest X-rays and Alzheimer’s disease (AD) in brain MRIs using longitudinal data.
• To identify potential model biases and clinically relevant biomarkers through deformation field interpretation.

Proposed method

• DeFI-GAN uses a generative adversarial network (GAN) framework where a generator produces deformation fields to transform diseased images (x1) into modified images (x̂0) that appear healthy.
• The generator G maps an input image x1 to a vector field φ, and the modified image is computed via bilinear/trilinear interpolation: x̂0(p) = x1(p + φ(p)).
• The critic D is trained to distinguish real healthy images (x0) from generated modified images (x̂0), using the WGAN loss formulation to ensure training stability.
• A gradient penalty (LRegD) enforces the Lipschitz constraint required by WGAN, improving training convergence.
• A total variation regularization (LRegG) penalizes complex, unrealistic deformations, promoting smooth and anatomically plausible field outputs.
• The full optimization minimizes a combined loss: LG + λRegG·LRegG for the generator, and LD + λRegD·LRegD for the critic.

Experimental results

Research questions

• RQ1 Can deformation fields provide more accurate and spatially coherent interpretations of disease evidence in medical images than existing visual attribution methods?
• RQ2 Does DeFI-GAN improve longitudinal prediction performance compared to baseline methods like VA-GAN?
• RQ3 What clinically relevant biomarkers or unexpected biases can be revealed through deformation field analysis that are missed by difference map-based methods?
• RQ4 How do the deformation fields generated by DeFI-GAN compare in interpretability and anatomical plausibility to those from additive perturbation-based methods?
• RQ5 Can DeFI-GAN highlight subtle, non-intuitive anatomical changes associated with disease that are not traditionally emphasized in radiological reports?

Key findings

• DeFI-GAN outperformed the VA-GAN baseline in longitudinal image validation, achieving a mean NCC score of 0.825 on the COPD dataset and 0.781 on the ADNI dataset, indicating better alignment of modified images with healthy counterparts.
• The method produced smoother, more spatially coherent deformation fields than VA-GAN, which generated noisy and less realistic difference maps.
• DeFI-GAN successfully highlighted known COPD biomarkers such as tracheal narrowing (saber-sheath trachea), diaphragm elevation, and soft-tissue thickening, which were less visible in VA-GAN outputs.
• Unexpected but potentially clinically relevant changes were identified, including reduced gastric bubble size and increased clavicle position, which may reflect patient positioning biases or comorbidities like osteoporosis.
• The method revealed atrophy in the hippocampus, precuneus, cerebellum, and brainstem in AD, consistent with known neurodegenerative patterns, and highlighted subtle ventricular expansion.
• DeFI-GAN uncovered potential dataset biases, such as changes in lung apex volume and shoulder positioning, which may stem from acquisition artifacts or patient-related factors like coughing or mobility issues.