[Paper Review] Learning a Discriminative Feature Network for Semantic Segmentation

Changqian Yu, Jingbo Wang+4 more|arXiv (Cornell University)|Apr 25, 2018

Multimodal Machine Learning Applications33 references88 citations

TL;DR

Introduces a Discriminative Feature Network (DFN) with a Smooth Network for intra-class consistency and a Border Network for inter-class distinction, achieving state-of-the-art results on VOC 2012 and Cityscapes without post-processing.

ABSTRACT

Most existing methods of semantic segmentation still suffer from two aspects of challenges: intra-class inconsistency and inter-class indistinction. To tackle these two problems, we propose a Discriminative Feature Network (DFN), which contains two sub-networks: Smooth Network and Border Network. Specifically, to handle the intra-class inconsistency problem, we specially design a Smooth Network with Channel Attention Block and global average pooling to select the more discriminative features. Furthermore, we propose a Border Network to make the bilateral features of boundary distinguishable with deep semantic boundary supervision. Based on our proposed DFN, we achieve state-of-the-art performance 86.2% mean IOU on PASCAL VOC 2012 and 80.3% mean IOU on Cityscapes dataset.

Motivation & Objective

• Reframe semantic segmentation to emphasize intra-class consistency and inter-class distinction at a macroscopic level.
• Develop a network that simultaneously learns robust, discriminative features across scales and along semantic boundaries.
• Integrate global context, channel-wise attention, and explicit semantic boundary supervision to improve segmentation accuracy.

Proposed method

• Propose Smooth Network using a U/V-shaped backbone with global average pooling and a Channel Attention Block to select discriminative features across scales.
• Introduce Border Network with deep supervision and semantic boundary loss (focal loss) to enhance inter-class distinction at boundaries.
• Use Refinement Residual Blocks to unify channel dimensions and refine features across stages.
• Combine Smooth and Border Networks in a bidirectional, stage-wise refinement framework (top-down for segmentation, bottom-up for boundaries).
• Optimize with a joint loss L = ell_s + lambda * ell_b, balancing segmentation and boundary supervision.

Experimental results

Research questions

• RQ1 Can a macroscopic view of segmentation improve intra-class consistency and inter-class distinction compared to pixel-wise methods?
• RQ2 Do a Smooth Network and a Border Network, operating with global context and semantic boundaries, respectively, improve mean IoU on standard benchmarks?
• RQ3 What is the impact of channel attention, global pooling, deep supervision, and boundary supervision on final segmentation performance?

Key findings

• DFN achieves state-of-the-art results on PASCAL VOC 2012 with 86.2% mean IoU on the test set after COCO fine-tuning.
• DFN achieves 80.3% mean IoU on the Cityscapes test set.
• The Smooth Network with global pooling and Channel Attention Block significantly improves mean IoU (e.g., from 72.86% to 79.54% in ablations).
• The Border Network with semantic boundary supervision provides a further gain, improving accuracy (e.g., from 79.54% to 79.67%).
• Multi-scale inputs and horizontal flips (MS_Flip) further boost performance to 80.01–80.60% on VOC2012 test depending on configuration.
• DFN demonstrates robust improvements across datasets without requiring post-processing like DenseCRF.