[Paper Review] Centralized Information Interaction for Salient Object Detection

Jiangjiang Liu, Zhiang Liu+1 more|arXiv (Cornell University)|Dec 21, 2020

Visual Attention and Saliency Detection73 references4 citations

TL;DR

This paper proposes a centralized information interaction (CII) mechanism that enhances salient object detection by fusing multi-scale features through a shared, parameter-efficient module, avoiding spatial interpolation. By integrating a relative global calibration (RGC) module, the method achieves state-of-the-art performance on five benchmarks with only ~4% additional parameters and reduced FLOPs.

ABSTRACT

The U-shape structure has shown its advantage in salient object detection for efficiently combining multi-scale features. However, most existing U-shape based methods focused on improving the bottom-up and top-down pathways while ignoring the connections between them. This paper shows that by centralizing these connections, we can achieve the cross-scale information interaction among them, hence obtaining semantically stronger and positionally more precise features. To inspire the potential of the newly proposed strategy, we further design a relative global calibration module that can simultaneously process multi-scale inputs without spatial interpolation. Benefiting from the above strategy and module, our proposed approach can aggregate features more effectively while introducing only a few additional parameters. Our approach can cooperate with various existing U-shape-based salient object detection methods by substituting the connections between the bottom-up and top-down pathways. Experimental results demonstrate that our proposed approach performs favorably against the previous state-of-the-arts on five widely used benchmarks with less computational complexity. The source code will be publicly available.

Motivation & Objective

• Address the limitation of existing U-shape networks in cross-scale feature interaction due to direct, uncentralized connections between bottom-up and top-down pathways.
• Overcome the degradation of spatial information caused by spatial interpolation during multi-scale feature fusion.
• Design a parameter-efficient, end-to-end trainable module that enables global context-aware feature interaction across scales without interpolation.
• Demonstrate that centralized feature interaction significantly improves both semantic strength and spatial precision in salient object detection.
• Enable plug-and-play integration with existing U-shape-based salient object detection models without modifying backbone or pathway designs.

Proposed method

• Introduce a centralized information interaction (CII) strategy that replaces direct skip connections with a shared, multi-scale processing module applied to features from all stages.
• Process multi-scale features through a shared module that encodes cross-scale information directly into filters, eliminating the need for spatial interpolation.
• Propose a relative global calibration (RGC) module that captures scale-specific global context by modeling relative relationships between features at different scales.
• Design the RGC module to operate on multi-scale inputs without spatial upsampling or downsampling, preserving exact spatial locations.
• Enable end-to-end training of the entire network with minimal additional parameters and FLOPs.
• Support flexible integration with any U-shape-based salient object detection model by replacing only the skip connections between pathways.

Experimental results

Research questions

• RQ1 Can centralized cross-scale feature interaction improve the representation quality of salient object detection beyond standard U-shape skip connections?
• RQ2 Does avoiding spatial interpolation in multi-scale feature fusion lead to better preservation of spatial details and improved detection accuracy?
• RQ3 Can a shared, multi-scale module effectively capture global context across different receptive fields without spatial transformation?
• RQ4 How does the proposed RGC module compare to conventional attention or pooling mechanisms in modeling scale-relative global context?
• RQ5 To what extent can the CII and RGC modules be generalized across different backbone architectures and U-shape detection frameworks?

Key findings

• The proposed method achieves state-of-the-art performance on five widely used salient object detection benchmarks, including PASCAL-S, HKU-IS, and DUTS-TE.
• With ResNet-50, the method achieves an F-measure of 0.950 on DUTS-TE, outperforming the previous SOTA method ITSD (0.947) with 67% fewer additional parameters.
• The ResNet-18 version of the method surpasses most ResNet-50-based prior methods in terms of F-measure and mean F-score, demonstrating strong parameter efficiency.
• The method introduces only ~0.97M (4.1%) additional parameters over ResNet-50 and 4.74G FLOPs additional computation, significantly lower than prior SOTA methods.
• Visual comparisons show that the method produces more complete and precise saliency maps, especially for small or complex-shaped objects, without edge-specific supervision.
• Ablation studies confirm that both CII and RGC contribute significantly to performance gains, with RGC providing a strong boost even with minimal parameter overhead.