[Paper Review] Selectivity or Invariance: Boundary-aware Salient Object Detection

Jinming Su, Jia Li+3 more|arXiv (Cornell University)|Dec 25, 2018

Visual Attention and Saliency DetectionComputer Science50 references20 citations

TL;DR

This paper proposes a boundary-aware network with successive dilation (BANet) to address the selectivity-invariance dilemma in salient object detection: selective features at boundaries for fine-grained distinction and invariant features in interiors for holistic object detection. By integrating a boundary localization stream, an interior perception stream, and a transition compensation stream with an integrated successive dilation module, BANet achieves state-of-the-art performance on six benchmark datasets, outperforming 16 SOTA methods with significant gains in F-measure and MAE.

ABSTRACT

Typically, a salient object detection (SOD) model faces opposite requirements in processing object interiors and boundaries. The features of interiors should be invariant to strong appearance change so as to pop-out the salient object as a whole, while the features of boundaries should be selective to slight appearance change to distinguish salient objects and background. To address this selectivity-invariance dilemma, we propose a novel boundary-aware network with successive dilation for image-based SOD. In this network, the feature selectivity at boundaries is enhanced by incorporating a boundary localization stream, while the feature invariance at interiors is guaranteed with a complex interior perception stream. Moreover, a transition compensation stream is adopted to amend the probable failures in transitional regions between interiors and boundaries. In particular, an integrated successive dilation module is proposed to enhance the feature invariance at interiors and transitional regions. Extensive experiments on six datasets show that the proposed approach outperforms 16 state-of-the-art methods.

Motivation & Objective

• To address the inherent conflict in salient object detection where interior features must be invariant to appearance changes, while boundary features must be selective to subtle differences.
• To improve segmentation accuracy, especially at fine boundaries and in regions with large appearance variation within salient objects.
• To develop a unified deep learning framework that separately handles feature selectivity and invariance across different object regions.
• To enhance contextual feature extraction in interiors and transitional regions using a novel multi-path dilation module.

Proposed method

• The network employs a ResNet-50 backbone to extract shared features, which are then fed into three dedicated streams: boundary localization, interior perception, and transition compensation.
• The boundary localization stream uses a simple subnetwork to extract highly selective features for precise boundary detection.
• The interior perception stream uses a complex architecture with the integrated successive dilation (ISD) module to extract invariant features resistant to appearance variations.
• The transition compensation stream corrects potential errors in transitional regions between interiors and boundaries by refining feature representations.
• The ISD module enhances contextual aggregation through multiple parallel paths with varying dilation rates, improving feature invariance across diverse visual patterns.
• Final saliency maps are generated by adaptive fusion of the three stream outputs, forming a boundary-aware feature mosaic for accurate, holistic detection.

Experimental results

Research questions

• RQ1 How can a deep learning model simultaneously achieve high selectivity at object boundaries and high invariance in object interiors?
• RQ2 What architectural components are most effective for mitigating the selectivity-invariance dilemma in salient object detection?
• RQ3 Can a dedicated transition compensation stream improve performance in regions where feature requirements shift from invariance to selectivity?
• RQ4 To what extent does the integrated successive dilation module enhance feature invariance in complex visual contexts?

Key findings

• BANet outperforms 16 state-of-the-art methods across six benchmark datasets, demonstrating consistent superiority in both F-measure and MAE.
• The inclusion of the boundary localization stream alone reduces MAE and increases F-measure significantly, indicating strong performance gains from enhanced boundary selectivity.
• Adding the transition compensation stream to the IPS + BLS setting improves Fβ score by up to 1.5% on the HKU-IS dataset, confirming its effectiveness in correcting transitional region errors.
• Removal of the ISD module leads to a dramatic performance drop, with incorrect salient object predictions in complex scenes such as reflections and overlapping objects.
• Replacing ISD with ASPP or ASPP-M yields inferior results, proving that ISD’s multi-path information flow is essential for robust feature invariance.
• Qualitative results show that BANet produces more accurate, sharp boundaries and better captures salient objects as coherent wholes, even under large appearance variations.