[Paper Review] Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation

Yunchao Wei, Huaxin Xiao+4 more|arXiv (Cornell University)|May 11, 2018

Advanced Neural Network Applications40 references45 citations

TL;DR

The paper reuses multi-dilated convolutional blocks to generate dense object localization from image-level labels, enabling state-of-the-art weakly- and semi-supervised semantic segmentation on PASCAL VOC 2012.

ABSTRACT

Despite the remarkable progress, weakly supervised segmentation approaches are still inferior to their fully supervised counterparts. We obverse the performance gap mainly comes from their limitation on learning to produce high-quality dense object localization maps from image-level supervision. To mitigate such a gap, we revisit the dilated convolution [1] and reveal how it can be utilized in a novel way to effectively overcome this critical limitation of weakly supervised segmentation approaches. Specifically, we find that varying dilation rates can effectively enlarge the receptive fields of convolutional kernels and more importantly transfer the surrounding discriminative information to non-discriminative object regions, promoting the emergence of these regions in the object localization maps. Then, we design a generic classification network equipped with convolutional blocks of different dilated rates. It can produce dense and reliable object localization maps and effectively benefit both weakly- and semi- supervised semantic segmentation. Despite the apparent simplicity, our proposed approach obtains superior performance over state-of-the-arts. In particular, it achieves 60.8% and 67.6% mIoU scores on Pascal VOC 2012 test set in weakly- (only image-level labels are available) and semi- (1,464 segmentation masks are available) supervised settings, which are the new state-of-the-arts.

Motivation & Objective

• Motivate and address the gap in dense object localization under image-level supervision for weakly supervised segmentation.
• Propose a simple, generic approach using multi-dilated convolutional blocks to transfer discriminative knowledge to non-discriminative object regions.
• Enable dense localization maps to improve segmentation training in both weakly- and semi-supervised settings.

Proposed method

• Augment a standard classification network with multiple dilation-rate blocks to enlarge receptive fields at multiple scales.
• Use class activation maps (CAM) per block to create object localization maps.
• Propose an anti-noise fusion strategy by averaging localization maps from blocks with dilations (d=3,6,9) and adding the result to the d=1 map.
• Train segmentation models with dense localization maps as pseudo masks, using saliency for background cues.
• Provide learning objectives for weakly-supervised (image-level labels) and semi-supervised (mixed strong/weak) settings.

Experimental results

Research questions

• RQ1 Can dilated convolutional blocks with multiple dilation rates produce dense, reliable object localization from image-level supervision?
• RQ2 Does anti-noise fusion of multi-dilated localization maps improve segmentation performance under weak and semi-supervised settings?
• RQ3 How does the proposed localization approach affect state-of-the-art results on VOC 2012 under weakly- and semi-supervised regimes?

Key findings

• Achieves new state-of-the-art mIoU on Pascal VOC 2012 test set in weakly-supervised setting: 60.8% (image-level labels only).
• Achieves new state-of-the-art mIoU on Pascal VOC 2012 test set in semi-supervised setting: 67.6%.
• Dense localization maps generated by multi-dilated blocks, when fused with anti-noise strategy, substantially improve segmentation training compared to using a single dilation or naive averaging.
• The method attains 60.4% validation mIoU and 60.8% test mIoU in the weakly-supervised setting, and 65.7% validation mIoU and 67.6% test mIoU in semi-supervised experiments.