[Paper Review] Distance transform regression for spatially-aware deep semantic segmentation

Nicolas Audebert, Alexandre Boulch+2 more|arXiv (Cornell University)|Sep 4, 2019

Domain Adaptation and Few-Shot Learning52 references44 citations

TL;DR

The paper introduces a distance transform regression loss as a multi-task regularizer for semantic segmentation, teaching networks to predict distance maps in addition to class labels to produce sharper, more coherent segmentations.

ABSTRACT

Understanding visual scenes relies more and more on dense pixel-wise classification obtained via deep fully convolutional neural networks. However, due to the nature of the networks, predictions often suffer from blurry boundaries and ill-segmented shapes, fueling the need for post-processing. This work introduces a new semantic segmentation regularization based on the regression of a distance transform. After computing the distance transform on the label masks, we train a FCN in a multi-task setting in both discrete and continuous spaces by learning jointly classification and distance regression. This requires almost no modification of the network structure and adds a very low overhead to the training process. Learning to approximate the distance transform back-propagates spatial cues that implicitly regularizes the segmentation. We validate this technique with several architectures on various datasets, and we show significant improvements compared to competitive baselines.

Motivation & Objective

• Motivate improving spatial regularity in FCN-based semantic segmentation.
• Propose using signed distance transforms as a regression target to inject spatial cues.
• Enable end-to-end learning with minimal network modifications.
• Show generalization across diverse domains including urban, RGB-D, and aerial imagery.

Proposed method

• Compute a signed distance transform (SDT) of the ground-truth masks for each class and normalize to [-1, 1].
• Train a FCN in a cascaded multi-task setup to predict SDTs (regression) and perform standard segmentation (classification).
• Fuse the predicted SDTs with intermediate features through a concatenation-based fusion before the final softmax.
• Use an L1 loss for SDT regression and a cross-entropy loss for classification, balancing with a hyperparameter lambda.
• Clamp SDT predictions via hardtanh to keep outputs in [-1, 1].

Experimental results

Research questions

• RQ1 Does incorporating SDT regression as a multi-task objective improve semantic segmentation quality over baselines?
• RQ2 How does the SDT-based regularization interact with different architectures and datasets?
• RQ3 What is the robustness of the approach to hyperparameter settings and registration to diverse domains?
• RQ4 Is SDT regression alone beneficial or must it be combined with classification to yield gains?

Key findings

• The multi-task SDT regression yields significant improvements over strong baselines across multiple datasets and architectures.
• SDT regression alone is not sufficient; coupling it with classification via multi-task learning provides the gains.
• Across aerial and indoor datasets, the method improves boundary coherence and reduces classification noise.
• In aerial benchmarks, SDT improves IoU and building shape regularity, with consistent gains across cities.
• In RGB-D and urban scene datasets, the approach yields improved overall accuracy and segmentation quality compared to baselines.