[Paper Review] LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models

Hantao Zhang, Liu, Yuhe+5 more|arXiv (Cornell University)|Mar 21, 2024

Advanced MRI Techniques and Applications4 citations

TL;DR

LeFusion introduces lesion-focused diffusion models that improve cardiac MRI pathology synthesis by concentrating diffusion learning on lesion regions, preserving background integrity through forward-diffused context integration. It achieves state-of-the-art performance in synthetic data augmentation, boosting nnUNet segmentation accuracy on myocardial infarction and no-reflow lesions in DE-MRI.

ABSTRACT

Patient data from real-world clinical practice often suffers from data scarcity and long-tail imbalances, leading to biased outcomes or algorithmic unfairness. This study addresses these challenges by generating lesion-containing image-segmentation pairs from lesion-free images. Previous efforts in medical imaging synthesis have struggled with separating lesion information from background, resulting in low-quality backgrounds and limited control over the synthetic output. Inspired by diffusion-based image inpainting, we propose LeFusion, a lesion-focused diffusion model. By redesigning the diffusion learning objectives to focus on lesion areas, we simplify the learning process and improve control over the output while preserving high-fidelity backgrounds by integrating forward-diffused background contexts into the reverse diffusion process. Additionally, we tackle two major challenges in lesion texture synthesis: 1) multi-peak and 2) multi-class lesions. We introduce two effective strategies: histogram-based texture control and multi-channel decomposition, enabling the controlled generation of high-quality lesions in difficult scenarios. Furthermore, we incorporate lesion mask diffusion, allowing control over lesion size, location, and boundary, thus increasing lesion diversity. Validated on 3D cardiac lesion MRI and lung nodule CT datasets, LeFusion-generated data significantly improves the performance of state-of-the-art segmentation models, including nnUNet and SwinUNETR. Code and model are available at https://github.com/M3DV/LeFusion.

Motivation & Objective

• To address long-tail data imbalance and algorithmic bias in clinical cardiac MRI by generating diverse, realistic synthetic pathology.
• To overcome limitations in prior generative models that fail to preserve background quality while synthesizing lesions.
• To develop a data-driven, controllable method for synthesizing myocardial infarction and persistent microvascular obstruction in DE-MRI.
• To jointly model multiple lesion types and generate diverse lesion masks to improve data augmentation.
• To validate the method’s effectiveness in boosting downstream segmentation performance using state-of-the-art models like nnUNet.

Proposed method

• Redesign the diffusion learning objective to focus exclusively on lesion regions, reducing complexity and improving controllability.
• Integrate forward-diffused background contexts into the reverse diffusion process to preserve structural fidelity of normal anatomy.
• Propose a joint modeling strategy using multi-channel decomposition to handle multiple lesion types simultaneously.
• Develop a diffusion-based generative model for lesion masks to increase shape and texture diversity in synthetic data.
• Use a repainting-style framework inspired by RePaint, but adapted to prioritize lesion-specific learning and context preservation.
• Train and evaluate on the Emidec DE-MRI cardiac lesion dataset, using nnUNet as the downstream segmentation benchmark.

Experimental results

Research questions

• RQ1 Can lesion-focused diffusion models generate high-fidelity myocardial pathology in cardiac MRI while preserving background anatomy?
• RQ2 How does focusing diffusion learning on lesion regions improve controllability and synthesis quality compared to joint background-lesion learning?
• RQ3 To what extent can joint modeling of multiple lesion types (e.g., MI and PMO) enhance data diversity and downstream segmentation performance?
• RQ4 Can a learned diffusion model for lesion masks generate more diverse and realistic shapes than hand-crafted methods?
• RQ5 Does synthetic data generated by LeFusion effectively improve the performance of state-of-the-art segmentation networks like nnUNet?

Key findings

• LeFusion-J achieved the highest Dice score (88.7) on the PMO lesion class, outperforming LeFusion-S (85.2) and RePaint (83.1), demonstrating superior joint lesion modeling.
• With 33×5 synthetic normal cases (N”), the Dice score for MI increased to 92.1, showing consistent performance gains with data scale expansion.
• LeFusion-J achieved the highest average PSNR (28.4) and SSIM (0.91) compared to baseline methods, indicating superior image quality and structural similarity to real pathology.
• The diffusion-generated lesion masks were visually closer to real masks and exhibited greater shape diversity than hand-crafted masks.
• Cond-Diffusion and RePaint methods disrupted background structure, leading to lower Dice scores, especially for PMO lesions.
• Synthetic data augmented with LeFusion’s mask generation significantly improved segmentation performance, proving its effectiveness for data augmentation.