[Paper Review] Joint Salient Object Detection and Camouflaged Object Detection via Uncertainty-aware Learning

Aixuan Li, Jing Zhang+5 more|arXiv (Cornell University)|Jul 10, 2023

Visual Attention and Saliency DetectionComputer Science3 citations

TL;DR

This paper proposes a novel joint learning framework for salient object detection (SOD) and camouflaged object detection (COD) by modeling their inherent contradiction through uncertainty-aware learning. It introduces data interaction-based augmentation, a task-wise contrastive module, and uncertainty estimation to improve robustness and performance, achieving state-of-the-art results on benchmark datasets with informative uncertainty maps.

ABSTRACT

Salient objects attract human attention and usually stand out clearly from their surroundings. In contrast, camouflaged objects share similar colors or textures with the environment. In this case, salient objects are typically non-camouflaged, and camouflaged objects are usually not salient. Due to this inherent contradictory attribute, we introduce an uncertainty-aware learning pipeline to extensively explore the contradictory information of salient object detection (SOD) and camouflaged object detection (COD) via data-level and task-wise contradiction modeling. We first exploit the dataset correlation of these two tasks and claim that the easy samples in the COD dataset can serve as hard samples for SOD to improve the robustness of the SOD model. Based on the assumption that these two models should lead to activation maps highlighting different regions of the same input image, we further introduce a contrastive module with a joint-task contrastive learning framework to explicitly model the contradictory attributes of these two tasks. Different from conventional intra-task contrastive learning for unsupervised representation learning, our contrastive module is designed to model the task-wise correlation, leading to cross-task representation learning. To better understand the two tasks from the perspective of uncertainty, we extensively investigate the uncertainty estimation techniques for modeling the main uncertainties of the two tasks, namely task uncertainty (for SOD) and data uncertainty (for COD), and aiming to effectively estimate the challenging regions for each task to achieve difficulty-aware learning. Experimental results on benchmark datasets demonstrate that our solution leads to both state-of-the-art performance and informative uncertainty estimation.

Motivation & Objective

• To address the inherent contradiction between salient and camouflaged objects, where high saliency implies low camouflage and vice versa.
• To improve model robustness by reusing easy COD samples as hard samples for SOD through data-level contradiction modeling.
• To explicitly model task-wise correlation using a contrastive learning module that encourages divergent activation maps for SOD and COD on the same input.
• To enable difficulty-aware learning via uncertainty estimation, distinguishing between task uncertainty (SOD) and data uncertainty (COD).
• To achieve single-forward uncertainty estimation for both tasks, enhancing interpretability and model reliability.

Proposed method

• Introduces a 'data interaction as data augmentation' strategy, where easy samples from the COD dataset are repurposed as hard samples for SOD to improve robustness.
• Proposes a task-wise contrastive learning module that encourages distinct feature representations for SOD and COD on the same image, promoting cross-task representation learning.
• Applies foreground-cropping data augmentation specifically to the COD dataset, simulating real-world variations and improving detection of small or complex camouflaged objects.
• Designs a confidence estimation module using adversarial learning, where the discriminator is trained on residuals between predictions and ground truth to identify uncertain regions.
• Integrates uncertainty estimation via variance of multiple predictions under cropping, aligning with detectability and complexity of camouflaged regions.
• Employs a multi-task training strategy with scheduled contrastive learning (every 5 steps) and balanced loss weighting to mitigate conflicting objectives between SOD and COD.

Experimental results

Research questions

• RQ1 How can the inherent contradiction between salient and camouflaged objects be leveraged to improve model generalization in joint detection?
• RQ2 Can easy samples from the COD dataset serve as effective hard samples for SOD, and how does this data-level interaction improve performance?
• RQ3 How can task-wise contrastive learning effectively model the divergent feature requirements of SOD and COD without causing convergence issues?
• RQ4 To what extent can uncertainty estimation via prediction variance and adversarial learning improve detection reliability and interpretability?
• RQ5 What hyperparameter settings balance performance and uncertainty estimation quality in a joint SOD-COD framework?

Key findings

• The proposed data interaction strategy significantly improves SOD performance, with a 403-image subset of PASCAL VOC 2007 used as hard samples leading to better SOD results.
• Task-wise contrastive learning enhances SOD performance, as shown by the improvement from 'JSOD1' to 'JSOD2' in Table III, though it slightly degrades COD performance due to conflicting objectives.
• Foreground-cropping data augmentation for COD improves detection of small or complex camouflaged objects, with uncertainty maps closely aligning with region-level detectability.
• The uncertainty estimation module produces reliable, single-forward uncertainty maps that correlate with model difficulty, validated through residual-based discriminator training.
• The joint framework achieves state-of-the-art performance on benchmark datasets, demonstrating the effectiveness of uncertainty-aware, contradiction-aware learning.
• Hyperparameter tuning, including contrastive loss weight (0.1) and adversarial loss weight (1.0), achieves a balance between performance and uncertainty estimation quality.