[Paper Review] Achieving state-of-the-art performance in the Medical Out-of-Distribution (MOOD) challenge using plausible synthetic anomalies

Sergio Naval Marimont, Giacomo Tarroni|arXiv (Cornell University)|Aug 2, 2023

Anomaly Detection Techniques and ApplicationsComputer Science3 citations

TL;DR

This paper proposes a novel unsupervised anomaly detection method for medical imaging that improves synthetic anomaly generation by using random shapes and edge-smoothing to create more realistic, challenging anomalies. The approach, based on 3D U-Net with self-supervision and model ensembling, achieved state-of-the-art performance—ranking first in both sample-wise and pixel-wise tasks—on the MICCAI 2022 MOOD challenge for abdominal CT and brain MRI.

ABSTRACT

The detection and localization of anomalies is one important medical image analysis task. Most commonly, Computer Vision anomaly detection approaches rely on manual annotations that are both time consuming and expensive to obtain. Unsupervised anomaly detection, or Out-of-Distribution detection, aims at identifying anomalous samples relying only on unannotated samples considered normal. In this study we present a new unsupervised anomaly detection method. Our method builds upon the self-supervised strategy consisting on training a segmentation network to identify local synthetic anomalies. Our contributions improve the synthetic anomaly generation process, making synthetic anomalies more heterogeneous and challenging by 1) using complex random shapes and 2) smoothing the edges of synthetic anomalies so networks cannot rely on the high gradient between image and synthetic anomalies. In our implementation we adopted standard practices in 3D medical image segmentation, including 3D U-Net architecture, patch-wise training and model ensembling. Our method was evaluated using a validation set with different types of synthetic anomalies. Our experiments show that our method improved substantially the baseline method performance. Additionally, we evaluated our method by participating in the Medical Out-of-Distribution (MOOD) Challenge held at MICCAI in 2022 and achieved first position in both sample-wise and pixel-wise tasks. Our experiments and results in the latest MOOD challenge show that our simple yet effective approach can substantially improve the performance of Out-of-Distribution detection techniques which rely on synthetic anomalies.

Motivation & Objective

• Address the challenge of unsupervised medical image anomaly detection without relying on costly manual annotations.
• Improve the realism and diversity of synthetic anomalies to enhance OoD detection performance.
• Overcome limitations of existing methods that rely on high-gradient edges between real and synthetic anomalies.
• Develop a robust, generalizable method for detecting pathological and non-pathological anomalies in diverse medical imaging modalities.
• Achieve top performance in the MICCAI 2022 Medical Out-of-Distribution (MOOD) challenge using only normal training data.

Proposed method

• Generate synthetic anomalies using complex, random-shaped patches instead of standard cuboids to increase diversity.
• Apply edge smoothing between the foreign patch and original image to reduce high-gradient artifacts that models could exploit.
• Train a 3D U-Net segmentation network in a self-supervised manner to localize synthetic anomalies.
• Employ patch-wise training and sliding window inference to improve localization accuracy on 3D volumes.
• Use model ensembling to boost robustness and generalization across different anomaly types.
• Leverage standard 3D medical image segmentation practices, including data augmentation and standard loss functions, to enhance performance.

Figure 1: Texture (foreign patch) and shape components (mask) of the synthetic anomaly generation process.

Experimental results

Research questions

• RQ1 Can using random-shaped synthetic anomalies improve the generalization and realism of unsupervised anomaly detection in medical imaging?
• RQ2 Does edge smoothing between synthetic anomalies and background images lead to more robust and generalizable OoD detection models?
• RQ3 Can a simple, self-supervised approach with improved synthetic anomaly generation outperform existing state-of-the-art methods in real-world challenges?
• RQ4 How does the performance of the method vary across different imaging modalities, such as abdominal CT and brain MRI?
• RQ5 To what extent can hyper-parameter tuning for one modality be transferred to another, and what are the implications for cross-modality deployment?

Key findings

• The proposed method achieved first place in both sample-wise and pixel-wise anomaly detection tasks in the MICCAI 2022 MOOD challenge.
• The method ranked first in the abdominal CT dataset for both sample-wise and pixel-wise evaluation, and fourth in the brain MRI pixel-wise task.
• The method achieved an overall first-place ranking in the MOOD 2022 challenge due to strong performance across both modalities.
• Experiments on a synthetic validation set demonstrated a substantial performance improvement over baseline methods using standard cuboid anomalies.
• The performance gap between abdominal CT and brain MRI in the pixel-wise task suggests modality-specific hyper-parameter optimization is necessary.
• The method shows strong potential for clinical deployment as a real-time, cost-effective tool to assist radiologists in detecting anomalies and imaging artifacts.

Figure 2: Examples of augmented shapes for each category used in the anomaly generation process. Note that these are axial 2D slices of the actual 3D shapes used in our method.