[Paper Review] Robust Saliency Detection via Fusing Foreground and Background Priors

Kan Huang, Chunbiao Zhu+1 more|arXiv (Cornell University)|Nov 1, 2017

Visual Attention and Saliency Detection12 references3 citations

TL;DR

This paper proposes a robust bottom-up saliency detection framework that fuses foreground and background priors using surroundedness cues and geodesic distance refinement. By jointly leveraging superpixel-based foreground seeds from high-surroundedness regions and background priors from image borders, and refining the saliency map via geodesic distance weighting, the method achieves state-of-the-art performance with a mean AUC of 0.9446 on ASD and 0.7619 on DUT-OMRON, outperforming 11 SOTA methods.

ABSTRACT

Automatic Salient object detection has received tremendous attention from research community and has been an increasingly important tool in many computer vision tasks. This paper proposes a novel bottom-up salient object detection framework which considers both foreground and background cues. First, A series of background and foreground seeds are selected from an image reliably, and then used for calculation of saliency map separately. Next, a combination of foreground and background saliency map is performed. Last, a refinement step based on geodesic distance is utilized to enhance salient regions, thus deriving the final saliency map. Particularly we provide a robust scheme for seeds selection which contributes a lot to accuracy improvement in saliency detection. Extensive experimental evaluations demonstrate the effectiveness of our proposed method against other outstanding methods.

Motivation & Objective

• Address the limitation of existing saliency detection methods that rely solely on foreground or background priors.
• Improve accuracy and robustness in salient object detection by integrating complementary cues from both foreground and background.
• Develop a reliable seed estimation scheme that assigns confidence levels to foreground and background seeds based on surroundedness and statistical thresholds.
• Enhance saliency map uniformity and boundary precision through geodesic distance-based refinement.
• Achieve state-of-the-art performance in saliency detection without requiring training data, maintaining efficiency for real-time use.

Proposed method

• Extract foreground seeds using a surroundedness map derived from the BMS algorithm, with superpixels generated via SLIC to reduce noise and preserve structure.
• Define strong and weak foreground seeds using thresholds relative to the mean surroundedness value: strong seeds have $ S_p(i) \geq 2 \cdot \text{mean}(S_p) $, weak seeds have $ \text{mean}(S_p) \leq S_p(i) < 2 \cdot \text{mean}(S_p) $.
• Compute foreground saliency using a graph-based ranking method that propagates saliency scores from seeds based on intrinsic manifold structure.
• Extract background priors from the image border regions and compute a separate background saliency map using the same ranking mechanism.
• Fuse foreground and background saliency maps by combining elements above the average value in each map, then re-rank using the combined set as seeds.
• Refine the final saliency map using geodesic distance-based weighting: $ S_{final}(q) = \sum_j \delta_{qj} \cdot S_{com}(j) $, where $ \delta_{qj} = \exp\left(-\frac{d_g^2(p,i)}{2\sigma_c^2}\right) $, with $ d_g $ being the shortest path distance on the superpixel graph.

Experimental results

Research questions

• RQ1 Can combining foreground and background priors improve the robustness and accuracy of bottom-up saliency detection compared to methods relying on only one prior?
• RQ2 How effective is the surroundedness cue in identifying reliable foreground seeds for saliency detection?
• RQ3 Does a confidence-based seed estimation scheme (strong vs. weak seeds) enhance detection performance compared to fixed thresholding?
• RQ4 Can geodesic distance-based refinement uniformly enhance saliency regions and improve boundary localization?
• RQ5 To what extent does the proposed fusion and refinement framework outperform existing state-of-the-art methods on benchmark datasets?

Key findings

• The proposed method achieves a mean AUC of 0.9446 on the ASD dataset, outperforming all 11 compared SOTA methods, including BMS, CAS, and wCtr.
• On the more challenging DUT-OMRON dataset, the method achieves a mean AUC of 0.7619, surpassing all baselines, including GBVS, ITTI, and PCA.
• The method achieves the lowest mean absolute error (MAE) of 0.0596 on ASD and 0.1068 on DUT-OMRON, indicating high precision in saliency map estimation.
• Visual comparisons show that the method produces more uniformly highlighted salient objects with better boundary preservation than competing models.
• The geodesic distance refinement step significantly improves saliency map uniformity, especially in enhancing the interior regions of salient objects.
• The method maintains high efficiency due to its training-free, bottom-up design, making it suitable for real-time applications.