[Paper Review] Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation

Saeid Asgari Taghanaki, Yefeng Zheng+6 more|arXiv (Cornell University)|May 7, 2018

Radiomics and Machine Learning in Medical Imaging4 citations

TL;DR

This paper proposes Combo Loss, a novel loss function that jointly addresses input and output class imbalance in multi-organ medical image segmentation by combining Dice loss and weighted cross-entropy in a curriculum learning framework. It enables explicit control over the false positive/negative trade-off during training, leading to faster convergence and state-of-the-art performance across PET, MRI, and ultrasound datasets, with improvements of up to 10.8% in Dice score over Focal Loss.

ABSTRACT

Simultaneous segmentation of multiple organs from different medical imaging modalities is a crucial task as it can be utilized for computer-aided diagnosis, computer-assisted surgery, and therapy planning. Thanks to the recent advances in deep learning, several deep neural networks for medical image segmentation have been introduced successfully for this purpose. In this paper, we focus on learning a deep multi-organ segmentation network that labels voxels. In particular, we examine the critical choice of a loss function in order to handle the notorious imbalance problem that plagues both the input and output of a learning model. The input imbalance refers to the class-imbalance in the input training samples (i.e., small foreground objects embedded in an abundance of background voxels, as well as organs of varying sizes). The output imbalance refers to the imbalance between the false positives and false negatives of the inference model. In order to tackle both types of imbalance during training and inference, we introduce a new curriculum learning based loss function. Specifically, we leverage Dice similarity coefficient to deter model parameters from being held at bad local minima and at the same time gradually learn better model parameters by penalizing for false positives/negatives using a cross entropy term. We evaluated the proposed loss function on three datasets: whole body positron emission tomography (PET) scans with 5 target organs, magnetic resonance imaging (MRI) prostate scans, and ultrasound echocardigraphy images with a single target organ i.e., left ventricular. We show that a simple network architecture with the proposed integrative loss function can outperform state-of-the-art methods and results of the competing methods can be improved when our proposed loss is used.

Motivation & Objective

• To address the dual challenge of input imbalance (few foreground voxels) and output imbalance (false positives vs. false negatives) in deep multi-organ segmentation.
• To develop a loss function that enables explicit, learnable control over the trade-off between false positives and false negatives during training.
• To improve segmentation performance on highly imbalanced medical imaging data across diverse modalities, including PET, MRI, and ultrasound.
• To provide a plug-in loss function that enhances existing architectures without architectural changes, improving generalization and convergence.
• To reduce reliance on post-processing by directly penalizing false positives and negatives through a differentiable, adaptive loss.

Proposed method

• The Combo Loss combines Dice similarity coefficient (DSC) and weighted cross-entropy (WCE) in a single differentiable loss function to stabilize training and avoid bad local minima.
• The loss function is formulated as: L_combo = α × (1 - DSC) + (1 - α) × WCE, where α controls the balance between DSC and WCE terms.
• A second hyperparameter β is introduced in the WCE term to explicitly control the trade-off between false positive and false negative penalties.
• The method employs curriculum learning by gradually increasing the influence of the WCE term to improve optimization stability and convergence speed.
• The loss is applied end-to-end in encoder-decoder networks (e.g., 3D U-Net, V-Net) without architectural modifications.
• Hyperparameters α and β are tuned via cross-validation to optimize performance on each dataset.

Experimental results

Research questions

• RQ1 Can a unified loss function effectively handle both input and output imbalance in multi-organ segmentation?
• RQ2 How does combining Dice and weighted cross-entropy improve model convergence and performance on imbalanced medical image data?
• RQ3 To what extent can the trade-off between false positives and false negatives be controlled via a learnable hyperparameter in the loss function?
• RQ4 Does the proposed Combo Loss outperform existing losses like cross-entropy, Dice, and Focal Loss across diverse imaging modalities?
• RQ5 Can the Combo Loss be used as a plug-in replacement to improve existing segmentation networks without architectural changes?

Key findings

• On the PET dataset, Combo Loss reduced false positive rate by 34.6% and false negative rate by 27.8% compared to standard cross-entropy loss.
• On the MRI dataset, Combo Loss improved Dice score by 2.3% and reduced false negative rate by 47.4% compared to Focal Loss when used with 3D V-Net.
• On the ultrasound dataset, Combo Loss achieved a 10.8% improvement in Dice score over Focal Loss when applied to 3D U-Net.
• The optimal β value for minimizing false positives and negatives was found to be 0.7 for ultrasound and 0.6 for MRI, demonstrating the effectiveness of hyperparameter control.
• When replacing original losses in U-Net and V-Net, Combo Loss increased Dice scores from 0.87 to 0.91 (MRI) and 0.85 to 0.92 (ultrasound), showing consistent gains.
• The method achieved faster convergence than standard cross-entropy and reduced outlier performance in Hausdorff distance, indicating more robust segmentation.