[Paper Review] Data efficient deep learning for medical image analysis: A survey

Suruchi Kumari, Pravendra Singh|arXiv (Cornell University)|Oct 10, 2023

AI in cancer detection4 citations

TL;DR

This survey provides a comprehensive, systematic review of data-efficient deep learning methods for medical image analysis, categorizing them into five supervision levels—no, inexact, incomplete, inaccurate, and limited supervision—based on label availability and quality. It synthesizes over 250 recent papers (2020–2023), analyzes key techniques like self-supervised learning, active learning, and federated learning, and identifies future research directions such as label-efficient FL and text-augmented learning.

ABSTRACT

The rapid evolution of deep learning has significantly advanced the field of medical image analysis. However, despite these achievements, the further enhancement of deep learning models for medical image analysis faces a significant challenge due to the scarcity of large, well-annotated datasets. To address this issue, recent years have witnessed a growing emphasis on the development of data-efficient deep learning methods. This paper conducts a thorough review of data-efficient deep learning methods for medical image analysis. To this end, we categorize these methods based on the level of supervision they rely on, encompassing categories such as no supervision, inexact supervision, incomplete supervision, inaccurate supervision, and only limited supervision. We further divide these categories into finer subcategories. For example, we categorize inexact supervision into multiple instance learning and learning with weak annotations. Similarly, we categorize incomplete supervision into semi-supervised learning, active learning, and domain-adaptive learning and so on. Furthermore, we systematically summarize commonly used datasets for data efficient deep learning in medical image analysis and investigate future research directions to conclude this survey.

Motivation & Objective

• Address the critical challenge of limited, expensive, and time-consuming annotations in medical image analysis due to privacy, expert availability, and data scarcity.
• Systematically categorize data-efficient deep learning methods based on the level and quality of supervision, from no supervision to only limited supervision.
• Provide a comprehensive, up-to-date review of recent advances (2020–2023) across 250+ papers in top-tier medical imaging venues and journals.
• Identify and analyze emerging techniques such as self-supervised pre-training, neural architecture search, and federated learning for data-efficient learning.
• Outline future research directions, including label-efficient federated learning, text supervision, and flexible model design, to advance the field.

Proposed method

• Categorize data-efficient methods into five supervision levels: no supervision, inexact supervision, incomplete supervision, inaccurate supervision, and only limited supervision.
• Subdivide each category into subcategories: e.g., inexact supervision includes multiple instance learning and weakly supervised learning; incomplete supervision includes semi-supervised learning, active learning, and domain adaptation.
• Review self-supervised learning techniques such as contrastive learning, predictive and generative self-supervision, and masked autoencoders for representation learning from unlabeled data.
• Analyze the role of pre-training on large-scale unlabeled medical images (e.g., using SwinUNETR or unified pre-training) to improve fine-tuning on small labeled datasets.
• Examine emerging paradigms like federated learning to train models collaboratively across institutions while preserving data privacy and addressing label scarcity in local datasets.
• Explore the integration of textual data (e.g., clinical reports, metadata) as auxiliary supervision to improve model generalization and reduce annotation dependency.

Experimental results

Research questions

• RQ1 How can deep learning models be made more data-efficient in medical image analysis when large, fully annotated datasets are unavailable?
• RQ2 What are the key methodological categories of data-efficient learning, and how do they differ in terms of supervision levels and practical applicability?
• RQ3 To what extent do self-supervised and weakly supervised methods reduce reliance on expert-annotated data in medical image segmentation, classification, and detection?
• RQ4 How can federated learning be adapted to handle label scarcity and heterogeneity across medical institutions while preserving privacy?
• RQ5 What role can textual data and multi-modal learning play in reducing annotation burden and improving model performance in low-data regimes?

Key findings

• The survey identifies five primary categories of data-efficient learning—no supervision, inexact, incomplete, inaccurate, and only limited supervision—offering a unified taxonomy for recent methods.
• Self-supervised pre-training on large-scale unlabeled 2D or 3D medical images significantly improves fine-tuning performance on small labeled datasets, as demonstrated by models like Self-Supervised SwinUNETR.
• Federated learning enables collaborative model training across institutions without sharing raw data, but current methods still rely heavily on supervised learning, limiting their use in low-label settings.
• Active learning and domain-adaptive learning are effective in reducing annotation costs by prioritizing informative samples and aligning distributions across domains.
• Neural architecture search (NAS) shows promise for automating model design but remains underexplored in data-efficient MIA, especially for segmentation and detection tasks.
• Text supervision, such as using clinical reports or metadata, enhances visual representation learning and is a promising direction for future research to reduce annotation dependency.