[Paper Review] On Mathews Correlation Coefficient and Improved Distance Map Loss for Automatic Glacier Calving Front Segmentation in SAR Imagery

Amirabbas Davari, Saahil Islam+4 more|arXiv (Cornell University)|Feb 16, 2021

Cryospheric studies and observations52 references37 citations

TL;DR

This paper proposes using the Mathews Correlation Coefficient (MCC) as an early stopping criterion and an improved distance map-based binary cross-entropy (BCE) loss to address severe class imbalance in deep learning-based semantic segmentation of glacier calving fronts from SAR imagery. The MCC-based early stopping improves Dice coefficient by 15% over standard BCE, and the modified distance map loss yields an additional 2% improvement, significantly enhancing segmentation accuracy on imbalanced data.

ABSTRACT

The vast majority of the outlet glaciers and ice streams of the polar ice sheets end in the ocean. Ice mass loss via calving of the glaciers into the ocean has increased over the last few decades. Information on the temporal variability of the calving front position provides fundamental information on the state of the glacier and ice stream, which can be exploited as calibration and validation data to enhance ice dynamics modeling. To identify the calving front position automatically, deep neural network-based semantic segmentation pipelines can be used to delineate the acquired SAR imagery. However, the extreme class imbalance is highly challenging for the accurate calving front segmentation in these images. Therefore, we propose the use of the Mathews correlation coefficient (MCC) as an early stopping criterion because of its symmetrical properties and its invariance towards class imbalance. Moreover, we propose an improvement to the distance map-based binary cross-entropy (BCE) loss function. The distance map adds context to the loss function about the important regions for segmentation and helps accounting for the imbalanced data. Using Mathews correlation coefficient as early stopping demonstrates an average 15% dice coefficient improvement compared to the commonly used BCE. The modified distance map loss further improves the segmentation performance by another 2%. These results are encouraging as they support the effectiveness of the proposed methods for segmentation problems suffering from extreme class imbalances.

Motivation & Objective

• To address the challenge of severe class imbalance in semantic segmentation of glacier calving fronts from SAR imagery.
• To improve generalization and prevent overfitting in deep neural networks trained on imbalanced data.
• To enhance segmentation performance by replacing standard BCE loss with a context-aware, distance map-augmented loss function.
• To evaluate and compare direct front-line prediction versus post-processing of segmented ice zones for calving front delineation.

Proposed method

• Uses the Mathews Correlation Coefficient (MCC) as an early stopping criterion to monitor validation performance, leveraging its invariance to class imbalance.
• Introduces a modified distance map loss that assigns higher weights to pixels near the calving front boundary, improving focus on minority-class regions.
• Applies the distance map to the binary cross-entropy loss function, using a learnable parameter k to control the influence radius of the distance penalty.
• Employs U-Net architecture for semantic segmentation of calving front positions in SAR images.
• Trains models using BCE with and without class weighting, MCC-based early stopping, and the improved distance map loss for comparison.
• Compares two segmentation strategies: direct prediction of calving front lines versus prediction of ice zones followed by post-processing to extract front boundaries.

Experimental results

Research questions

• RQ1 Can MCC serve as a more reliable early stopping criterion than BCE loss in highly imbalanced segmentation tasks?
• RQ2 Does incorporating a distance map into the BCE loss function improve segmentation performance on imbalanced calving front data?
• RQ3 How does MCC-based early stopping compare to BCE-based early stopping in terms of Dice coefficient and IOU?
• RQ4 What is the relative performance of direct front-line prediction versus post-processed zone segmentation for calving front detection?
• RQ5 Does the improved distance map loss reduce false positives and improve boundary localization compared to standard BCE?

Key findings

• Using MCC as the early stopping criterion improves the Dice coefficient by an average of 15% compared to standard BCE-based early stopping.
• The improved distance map BCE loss further increases the Dice coefficient by approximately 2% beyond the MCC-based improvement.
• The MCC-monitored model shows significantly fewer false positives and more accurate boundary localization, especially in regions with high class imbalance.
• The distance map loss with optimal k (k=5) outperforms both standard BCE and k=1 variants, particularly at higher tolerance thresholds (e.g., 155 m for AP lines).
• The direct prediction of calving front lines yields more accurate but slightly discontinuous results, while post-processed zone segmentation produces continuous boundaries but may include non-calving-front features like bedrock.
• The combination of MCC-based early stopping and improved distance map loss achieves the highest performance across all metrics (IOU, Dice, MCC) on both AP and Jakobshavn Isbræ datasets.